Implementation
This documentation gives you an insight into the structure of the ZITADEL database. The goal is to give you a rough overview, so you know where which data is stored and which database schemas and tables are used.
Event
The single source of truth of ZITADEL are the events that are stored in the eventstore. From these events all different kind of resources e.g Users, Projects, Applications, etc. can be computed.
An event has the following data:
| Attribute | Description | Example |
|---|---|---|
| id | unique identifier of the event, this is generated by the database | b6402a60-e4655-4cc0-904b-f5c6760c4406 |
| aggregate_type | the type of the aggregate, an aggregate can be compared to a resource or an object. One aggregate contains multiple event types | user |
| aggregate_id | The unique identifier of an aggregate, this is generated by ZITADEL and is a sonyflake id | 168096909691353697 |
| aggregate_version | The aggregate version shows in which version of the aggregate the event was created. This is needed to be able to compute correct objects | v1 |
| event_type | The type of the event | user.human.added |
| event_sequence | The event sequence is a sequence number that is incremented by one for each event on the instance. For technical reasons, a number can be omitted in some cases. This is needed so that the sequence of the events can be ensured. | 1234 |
| previous_aggregate_sequence | This number is the sequence of the event last created on this specific aggregate. E.g. Last user with specific aggregate_id | 1233 |
| previous_aggregate_type_sequence | This number is the sequence of the event last created on this aggregate. E.g Last User | 1230 |
| creation_date | timestamp when the event was created | 2022-07-05 13:57:56.358774+00 |
| editor_user | The editor user contains mostly an unique identifier of a user. And tells who did the request that led to this event. Sometimes this can also be a name of a system within ZITADEL. | 165460784409638965, NOTIFICATION, LOGIN |
| editor_service | The service defines which API was called when the event got created. If the event was created from the system itself this is empty. | Admin-API |
| resource_owner | The resource owner defines to which organization/resource_owner the event belongs. This is an id generated by ZITADEL as sonyflake id | 168051083313153168 |
| instance_id | ZITADEL is capable of containing multiple ZITADEL instances withing the system. This id is the unique identifier of the Instance and is generated by ZITADEL as sonyflake id. | 165460784409737865 |
Schemas
| Schema | Description | Examples |
|---|---|---|
| System | The system contains everything that is needed outside the ZITADEL instances. | assets, encryption_key |
| Eventstore | Eventstore is the base of ZITADEL and is the single source of truth. All the events stored in the eventstore can be used to generate different projections. | events, instance sequences, system wide unique constraints |
| Projections | The projections contain all the computed objects which are used for reading requests. | users, projects, etc |
| Auth | This contains projections which are used for the auth api. All projections in this schema should be moved to Projections soon | users, auth_request, etc. |
| Adminapi | This contains projections which are used for the admin api. All projections in this schema should be moved to Projections soon | styling |
| Notification | This contains projections which are used for sending notification. All projections in this schema should be moved to Projections soon | styling |
Projections
The projections in ZITADEL contain all the computed objects, that are used for the reading requests. It is possible that the projections are slightly behind the actual event and not all objects are up-to-date.
Pub-Sub
To keep the projections as up-to-date as possible, an internal pub-sub system is used. As soon as an event is written to the event store, it is sent to the projections that have subscribed to this aggregate.
Spooler
It is sometimes possible for technical reasons that not all events were sent to the projections. For this reason, a spooler runs in parallel, which checks every n minutes whether there are new events that have not yet been processed.
Current Sequence
To ensure that no events get missed when creating the Projections, ZITADEL stores the current sequence, that was processed. You can find the current sequence in the following tables:
- projections.current_sequences
- notification.current_sequences
- auth.current_Sequences
- adminapi.current_sequences
The current sequence is stored for each ZITADEL instance and table.
| Attribute | Description | Examples |
|---|---|---|
| projection_name | The name of the projection for which the sequence is valid. | projection.users |
| aggregate_type | The aggregate type where the sequence was from | user |
| current_sequence | The sequence that was last processed | 1234 |
| instance_id | The instance to which the event belongs | 165460784409737834 |
| timestamp | Timestamp when the table was updated | 2022-07-05 13:57:59.454798+00 |
Failed Events
Sometimes an event cannot be processed correctly for some reason and an error occurs. The event is then tried to be processed n times. When a defined number of attempts have failed, the event is stored in the Failed Events Table and the next event is processed. This must be done so that the projection is not blocked and no further events are processed.
You can find the failed_events in the following tables:
- projections.failed_events
- notification.failed_events
- auth.failed_events
- adminapi.failed_events
| Attribute | Description | Examples |
|---|---|---|
| projection_name | The name of the projection for which the failed event should have been processed. | projection.users |
| failed_sequence | The sequence of the event that failed | 1234 |
| failure_count | The number of times the event was attempted to be processed. If the number is lower than the max. attempts the event could be processed, but not on the first attempt | 5 |
| error | The error message that occurred when the event could not be processed | User not found |
| instance_id | The instance to which the event belongs | 165460784409737834 |