3. CORAL Architecture

3.1. Primary Use Cases and Software Requirements

CORAL has been developed in order to provide technology independent access to relational databases for the various components of POOL, the Conditions Database project (COOL) and several applications and components of the software frameworks of the LHC experiments. The access patterns that are possible through the CORAL API reflect the needs of the applications of the LHC experiments.

In order to achieve the maximum insulation from the technology-specific implementations, we have chosen to minimize the SQL exposed through the public API of CORAL to the clients. Given that the CORAL API is a set of C++ interfaces (mainly pure abstract classes), no SQL types are exposed to the user for the description of tables, views and query result sets. The SQL to C++ type mapping (and vice versa) is handled by the technology-specific implementation component. The need for the maximal SQL insulation from the client software can be demonstrated simply showing the SQL statements for the two mostly used technologies (MySQL and Oracle) in rather simple tasks:

  • Creation of a table
    • MySQL SQL syntax:
      CREATE TABLE T_t ( I BIGINT, X DOUBLE )
      
    • Oracle SQL syntax:
      CREATE TABLE "T_t" ( I NUMBER(20), X BINARY_DOUBLE )
      
  • Query fetching only the first rows of the result set
    • MySQL SQL syntax:
      SELECT X FROM T_t ORDER BY I LIMIT 5
      
    • Oracle SQL syntax:
      SELECT * FROM (SELECT X FROM "T_t" ORDER BY I) WHERE ROWNUM < 5
      

The CORAL approach is to present an identical API for such cases, where from the functionality point of view are identical (among the technologies), but the necessary SQL statement is significantly different.

The design of CORAL addresses the requirement of being able to use different authentication mechanisms (user/password pairs, certificates) when connecting to a database schema. Moreover, it takes into account the fact that databases are addressed logically in a user application, which means that there might be a level of indirection before connecting to an actual database.

Finally, in a distributed environment there is always a need for some client-side monitoring of the operations with a database. CORAL had to allow the software plugging of adapters which send the relevant information to existing monitoring frameworks.

3.2. Architectural Choices

CORAL has adopted a component-based architecture for several reasons:

  • Minimization of the compile-time dependencies of any client software component.
  • Implementation of the insulation layer in terms of abstract interfaces.
  • Enabling the parallel and independent development of the various CORAL components, hence more efficient unit and integration testing.
  • Allowing for multiple implementations of the same interfaces.
  • Allowing for the self-consistent deployment of only a subset of the full CORAL library set.

The realization of the component architecture has been based on the plugin management and the component framework of the SEAL project. This allows the development, testing and deployment of new implementation components without any single change in existing libraries and the configuration. Moreover, the SEAL component framework and its context hierarchy in particular, allow the various implementation components to make use of the functionality of other components exclusively through the relevant abstract interfaces without selecting by value (i.e. by specifying the corresponding logical name) any particular implementation. Instead, a component performs a search in its context hierarchy for any implementation of a particular interface.

The requirement of flexible authentication mechanisms in a secure, distributed and grid-controlled environment had a consequence that a connection to a database is attempted, the authentication credentials should not be provided directly by the client software. Instead, appropriate authentication modules have to be invoked to provide the necessary credentials depending on the "database role" of the user and the read/write accessibility of the particular database. This lead to a design of the CORAL API with decoupled sets of interfaces for database connection and for the retrieval of the authentication credentials. The decoupled interface sets were therefore implemented by completely decoupled components.

The same approach has been followed for the functionalities of client-side monitoring and database service lookup. This lead naturally to an architecture with the minimum number of dependency levels among the software components.

Most implementation components implement a set of the public interfaces. There are also cases where components implement "developer-level" interfaces, which are simply abstract classes, not useable by the client software, but only by the various CORAL components. This is the case for example of the components implementing the interfaces related to the client-side monitoring, where the interfaces are called only by the components implementing the RDBMS-specific functionality. This can be shown schematically in the following picture:

It has to be noted that there is also a rather granular class hierarchy of C++ exceptions that are thrown in case of error conditions. The granularity in the exception hierarchy allows the client code to follow different recovery procedures depending on the source of the error and the conditions under which it occurs.

3.3. Static view of the architecture

The CORAL packages and their dependencies are shown in the following picture:

The public API of CORAL is defined in the CoralBase and RelationalAccess packages. The former contains the definitions of the name/type/value structures (AttributeList and related classes) which are used as row buffers for writing and reading data from a database and a Blob C++ type which is used for I/O with table columns of SQL type BLOB. The latter contains the user- and developer-level interfaces (abstract classes) which expose the full functionality of CORAL. A client software component is expected to include the header files only from these two packages and link against the two corresponding libraries.

The interfaces defined in RelationalAccess are implemented in several component libraries which are loaded at run time and only at demand by the SEAL plugin management mechanism. Shared implementation code is kept in the CoralCommon package. With this arrangement we have achieved to minimize the number of dependency levels and code duplication across the various packages.

Currently there are

  • four RDBMS-specific implementations for the interface set related to the functionality of accessing data in a relational database (Oracle, SQLite, MySQL and Frontier),
  • two implementations of the interface set related to the retrieval of authentication credentials (one based on XML files and another on environment variables),
  • an implementation of the interface set responsible for peforming the necessary technology dispatching given a connection string (RelationalService),
  • two implementations of the interface set responsible for peforming logical to physical database service lookup operations (one based on XML files and another on LFC),
  • an implementation based on LFC for both the lookup and authentication service funtionality,
  • a simple implementation of the interface set responsible for registering and reporting monitoring events generated by the RDBMS plugins (MonitoringService),
  • an implementation of the interface set responsible for managing open connections on the client and for the overall system configuration (ConnectionService)

In addition to the above, there is also the PyCoral package, which provides a coral python module which exposes the CORAL User API in python.

The CORAL architecture allows the development of additional or alternative implementations of the base interfaces which can work immediatelly with the rest of the system. The only necessary requirement in order to achieve that is the strict compliance to the interface semantics.