Natalia Castiglioni

Introduction

At GumGum, many of our applications must keep a record of what operations were done in the database (triggered by the API or the application). Most commonly, this implies keeping track of each user's interaction with the system, including metadata about the type, time, and trigger of the interaction. We sometimes refer to this type of logging as an audit trail and consider it a crosscutting concern that must be addressed while designing the system.

 

 

ActivityLogService

When I began working on the initial phases of our Spring based Java API for serving our advertising related applications (known internally as Advertising API), we had implemented a traditional approach of using a specially purposed Service class whose sole responsibility was managing the audit log. This service would simply dump the fields passed to it into one of our audit logging tables. In every other method in every other service that manipulated the database, we would tack on a call to this Activity Log Service and pass along metadata describing the activity that just took place.

The usage was pretty straightforward… basically, we have to do the following for each database operation:

Create Operation:

activityLogService.logCreateOperation(
  zoneFrequencyCap.getTrackingId(),
  ZONE_FREQUENCY_CAP_TABLENAME_FOR_ACTIVITY_LOG,
  activityLogService.toLogValue(zoneFrequencyCap));

Update Operation:

activityLogService.logUpdateOperation(
  zoneFrequencyCap.getTrackingId().toString(),
  ZONE_FREQUENCY_CAP_TABLENAME_FOR_ACTIVITY_LOG,
  activityLogService.toLogValue(previous),
  activityLogService.toLogValue(zoneFrequencyCap));

Delete Operation:

activityLogService.logDeleteOperation(
  request.getTrackingId(),
  ZONE_FREQUENCY_CAP_TABLENAME_FOR_ACTIVITY_LOG,
  activityLogService.toLogValue(zoneFrequencyCap));

 

This worked well within the existing architecture, but some drawbacks of this mechanism became apparent quite quickly. For example, we frequently found that when creating or updating entities, we have to query the database twice -- the second time just to get the updated object so that the log is meaningful. Apart from that, the fact that each developer is responsible for calling that service is error prone since we may perform the call incorrectly or simply forget to do it at all. Lastly, maintaining this whole service plus the infrastructure that orbits it is definitely overhead for us engineers. I do not think it would be a stretch to say we should be focusing on solving business problems rather than bogging ourselves down in the minutiae of these crosscutting concerns that are solved out of the box in many modern frameworks.

However, one of the advantages of this approach is code readability. Any new engineer on the project shouldn’t take more than a few minutes to understand how auditing is done within the system, as you can easily visually dissect the part of any service method that is performing the auditing from the part that is performing business logic.

 

Spring Boot with JPA alternatives

After working on Advertising API for one rather large internal project, we decided to take what we learned from building the initial API and try starting from scratch with a V2 of the API based on Spring Boot and JPA. We identified our audit logging system as a pain point to improve, and began researching the available options to better implement the audit logging in a simpler and more maintainable way. Our goal was to improve the approach in terms of reducing overhead for developers, improving performance (ideally by avoiding multiple calls to the database), and reducing complexity.

When it comes to these technologies, there are a couple of quite distinct approaches, each with unique pros and cons, so let’s quickly review them.

 

JPA entity lifecycle callbacks

JPA defines entity lifecycle callbacks that allow us to use these callbacks to take certain actions before or after specific operations on the entity that’s being persisted, modified, or deleted.

There is a set of annotations that allow us to define which callbacks we want to implement on the entity, for example:

@PrePersist

protected void onCreate() {
  if (StringUtils.isEmpty(countryCode)) {
    countryCode = "US";
  }
}

Similarly, we can use PrePersist, PreUpdate, and PreRemove, along with their counterparts PostUpdate, PostCreate, PostRemove, and PostLoad. Each of these callbacks will be invoked during the particular entity lifecycle phase that their name alludes to.

These callbacks represent places where we can add minor logic, but, as seen in the previous example, they are not intended to be leveraged to perform database operations or start new transactions.Further, according to the documentation, the callbacks should not invoke EntityManager or Query operations, so we can conclude that we should not modify the entities in the same persistence context. (as cited from https://www.baeldung.com/database-auditing-jpa, “…the lifecycle method of a portable application should not invoke EntityManager or Query operations, access other entity instances, or modify relationships within the same persistence context. A lifecycle callback method may modify the non-relationship state of the entity on which it is invoked.”)

This shows us that callbacks can be very good to add or modify attributes like created/updated at or created/updated by, but not for saving a record for audit information into the database.

 

JPA entity listeners

Similar to the callbacks, we can centralize all of the behaviour in a single class and use the annotation @EntityListeners, as in the following example:

@Entity
@Table(name = "test")
@EntityListeners(AuditListener.class)

public class Test {
  @Id
  @GeneratedValue(strategy = GenerationType.AUTO)
  @Column(name = "id")
  private Integer id;
}

With this approach, all of the entities that requires audit logging can use the annotation @EntityListeners and concentrate the callbacks in a single class, AuditListener. It is essentially the same as callbacks, but with added flexibility to reuse code.

 

Hibernate Envers

The Hibernate framework offers Envers as a module implementing auditing and versioning of persistent classes; it only requires configuration and adding a single new dependency to the project.

The framework is simple to use, with an annotation (@Audited) to mark the entities that we want to audit . It uses the concept of revisions: each transaction could potentially create a revision on an entity if it is modified, and there is one historical table for the audited entity, so all of the historical information for each entity is stored there. This approach did not meet our use case since we already had a table where we store the information, and the focus on auditing is more action/user oriented, so we did not fully evaluate this technique.

 

Spring Data JPA Auditing

This approach allows us to persist the CreatedBy, CreatedDate, LastModifiedBy, and LastModifiedDate columns for any entity, assuming those four attributes are the auditing metadata for each entity table.

To achieve this, we only needed to set up minimal infrastructure and apply some basic custom configurations.

We added the following to the Spring configuration class to indicate that we are using this mechanism for auditing and provide an implementation of the AuditorAwareImpl:

A possible implementation for AuditorAwareImpl would be the following:

@Configuration
@EnableJpaAuditing

public class Config {
  @Bean
  AuditorAware<String> auditorProvider() {
    return new AuditorAwareImpl();
  }
}

public class AuditorAwareImpl implements AuditorAware<String> {
  @Override
  public Optional<String> getCurrentAuditor() {
    OktaUserDetails user = (OktaUserDetails) SecurityContextHolder.getContext().getAuthentication().getPrincipal();
    return Optional.ofNullable(user.getUsername());
  }
}

In this example, we are using a custom implementation to get the user name of the logged user (OktaUserDetails class).

After this, we can use the annotations provided to map the 4 fields for auditing: @CreatedAt, @LastModifiedBy, @LastModifiedDate, and @CreatedDate. In order to make it more maintainable, we can move this into a base class (using @MappedSuperClass), and this class would be extended by all the entities that require audit.

The main drawback to this is the lack of flexibility since we must use those fields on a per entity table basis and therefore cannot use another table for auditing. Another issue with this approach is that delete operations can not be tracked, as the 4 fields will be deleted when the record itself is deleted from the table.

 

Hibernate Interceptors

Hibernate Interceptors are classes defined to react to certain Hibernate events. There are two ways of implementing it: implementing the Interceptor interface or extending EmptyInterceptor. In general, we choose to extend EmptyInterceptor since the other approach will force you to implement all of the methods defined in Interceptor interface even though we normally would only use 2 or 3 of them.

Typical overridden methods for EmptyInterceptor are:

This is the approach we decided to implement in Advertising API v2 and is currently working in production.

 

Advertising API V2 Audit Logging Implementation

The implementation we currently have uses Hibernate EmptyInterceptor, and allows to save a record into activity_logs table each time a record in the database is modified. We do that by implementing onSave, onDelete, onFlushDirty and beforeTransactionCompletion operations.

The interceptor is registered at the application level in the JpaConfig class, which contains all the configuration relative to JPA:

properties.put("hibernate.session_factory.interceptor", activityLogInterceptor);

Keep in mind that they can be registered as a session factory or as a session interceptor depending on the desired use case and if we need to tie it to the session factory or a specific session.

We also define an Identifiable interface which is implemented by all entities and allows us to determine which entities we care enough about to log changes for. All the entities are auditable by default, that’s why we define a default implementation for isAuditable method, but in the case of an entity which is not auditable it can override that method to return false.

public interface Identifiable<T> {
  T getId();
  default boolean isAuditable() {
    return true;
  }
}

This approach has been working fine for some time now and is a vast improvement upon the original architecture of audit logging in the API.

 

Thanks to Collin & Michele who read this article and give me valuable feedback about this post.

 

References

https://www.baeldung.com/database-auditing-jpa

https://hibernate.org/orm/envers

https://www.baeldung.com/hibernate-interceptor

Guides