America/Sao_Paulo
Home
About
Work
Blog
Posts

Idempotence: Importance and role in building resilient applications

January 29, 2025
Cover foto.
 If you work in software development, you've probably come across the concept of idempotence. But what does it mean and how important is it in building fault-resilient applications? Let's explore this together!

The Concept of Idempotence

An operation is considered idempotent when its repeated execution produces the same final state in the application, regardless of the number of times it is called. This does not mean that the response will always be the same, but rather that subsequent calls will not have additional side effects. For example, a DELETE request on a resource in a REST API, if correctly implemented, should remove the resource on the first execution and return a status 204 No Content. Subsequent calls must return a 404 Not Found, as the resource no longer exists, keeping the application state unchanged after the first execution. Operations in REST APIs like GET, PUT, HEAD and OPTIONS, if implemented correctly, should also be idempotent, while others like POST should not, as with each request it is expected that a new resource is created, generating side effects. The PATCH method, however, is the subject of controversy regarding the presence or absence of idempotence. As it is up to the developer to assign the behavior of each HTTP call to an API, what was discussed is not always the rule.

How important is Idempotence?

When designing systems that are resilient to failures, it may be necessary to consider retry strategies in some flows. To achieve this, it is important that the operations performed do not lead the application to inconsistent states. Imagine receiving an error when making a payment and, when trying again, discovering that the amount was debited from your account twice? Or having an application that listens for messages from a queue using Amazon SQS, which may occasionally deliver a message more than once? You need to be sure that it is safe to retry, and that is where the importance of idempotency comes in! When calling an idempotent method more than once, we don't need to worry about inconsistent states generated by retries, since we know that there will be no side effects in subsequent calls. That's why it's important to know strategies for implementing idempotency even in cases where it's not considered by default, such as in the POST method we mentioned earlier, and that's what we'll cover next.

Implementing Idempotency

In some cases, it may be necessary to make a POST method idempotent. We wouldn't want to create more than one account when making multiple identical requests to the server, would we? There are some methods that can be applied to make a POST method idempotent, such as using an idempotency key. The idempotency key is a field that we can add to the resource and that must be filled with information that makes that resource unique. In the case of creating an account mentioned above, we could use the CPF as the idempotency key. To ensure that two resources do not have the same idempotency key, we can create a rule in the application layer or make that field unique in the database layer, avoiding duplications. When we do not have an obvious field that makes that resource unique, as in the case of the CPF, a possible solution is to use a hash of the resource created with the values ​​of the fields of interest that make the resource unique. However, we must be careful when using this alternative, as it is possible that there may be hash collisions, preventing the publication of resources even if they are different. When idempotency needs to be guaranteed only for a certain period of time, we can use an in-memory database like redis to store the idempotency key, specifying an expiration time for this value. This way, we are less likely to create a problem due to a hash collision.

Example in Kotlin with Spring

To illustrate how it is possible to implement the strategies mentioned, I created a repository on github containing the code of an example application in Kotlin using Spring Boot 3.4.2.

Running the project

The project is very simple and to run it you just need to have installed java version 21 and docker and follow the instructions below:
  1. Clone the repository
git clone https://github.com/daanrsantiago/idempotency-example
  1. Navigate to the project directory:
cd idempotency-example
  1. Build the project using gradle
./gradlew build
  1. Run docker compose
docker compose up
  1. Run the application
java -jar build/libs/idempotency-example-0.0.1-SNAPSHOT.jar
After initialization, we can access http://localhost:8080/swagger-ui.html from the browser to start interacting with the application via the swagger-ui interface. The result should be something like the following:
Swagger-UI Page.
 From there, it is simple to interact with the application's resources.

Implementing the Publication and User Resources

The application defines two unrelated resources: Publication and User. Their representations in the database are defined by the following files:
Publication.kt
package dev.danielsantiago.idempotency.model

import jakarta.persistence.Entity
import jakarta.persistence.GeneratedValue
import jakarta.persistence.GenerationType.AUTO
import jakarta.persistence.Id
import java.time.LocalDateTime

@Entity
data class Publication(
  @Id
  @GeneratedValue(strategy= AUTO)
  var id: Long? = null,
  var title: String,
  var content: String,
  val creationDate: LocalDateTime = LocalDateTime.now()
)
User.kt
package dev.danielsantiago.idempotency.model

import jakarta.persistence.Column
import jakarta.persistence.Entity
import jakarta.persistence.GeneratedValue
import jakarta.persistence.GenerationType.AUTO
import jakarta.persistence.Id
import jakarta.persistence.Table

@Entity
@Table(name = "users")
data class User(
  @Id
  @GeneratedValue(strategy = AUTO)
  var id: Long? = null,
  val name: String,
  val age: Int,
  @Column(unique = true)
  val cpf: String
)
Note that the unique=true parameter was used in the definition of the cpf column of the User class. This means that there can only be users with different cpf values, in practice, the cpf acts as an idempotency key. When trying to save two users with the same cpf value, an exception is generated in the database layer, which the controller recognizes and returns a 422 error to the client. Now let's analyze the controller of the Publication resource
Publication.kt
package dev.danielsantiago.idempotency.controller

import ...

@RestController
@RequestMapping("/publication")
class PublicationController(
  val publicationRepository: PublicationRepository,
  val template: RedisTemplate<String, String>
) {
  val PREFIX = "publication:idempotency:"

  ...

  @PostMapping()
  fun publishPublication(@RequestBody body: PublicationRequest): ResponseEntity<Publication> {
      val publishedPublication = publicationRepository.save(
          Publication(
              title = body.title,
              content = body.content
          )
      )
      return ResponseEntity.status(CREATED).body(publishedPublication)
  }

  @PostMapping("/idempotent")
  fun eventualIdempotency(@RequestBody body: PublicationRequest): ResponseEntity<Publication> {
      if(template.opsForValue().setIfAbsent(PREFIX + body.hashCode(), "1", Duration.ofMinutes(1))!!) {
          val publishedPublication = publicationRepository.save(
              Publication(
                  title = body.title,
                  content = body.content
              )
          )
          return ResponseEntity.status(CREATED).body(publishedPublication)
      } else {
          return ResponseEntity.status(UNPROCESSABLE_ENTITY).build()
      }
  }
  ...
}
Two POST mappings were defined, one on the /publication path that allows duplications, and is therefore not idempotent, and another on the /publication/idempotent path, which does not allow duplications during a period of one minute, and is therefore idempotent during that period. When the application receives a request on the /publication/idempotent path, it tries to store a value in the "publication:idempotency:{PublicationHash}" key with an expiration period of 1 minute in our Redis instance. If this value does not exist, the application creates this value and saves the publication successfully, otherwise the publication is not saved and a 422 status is returned to the client. This implementation allows only publications with different information to be created within a period of 1 minute, as long as there are no collisions in the created hash. This can be interesting to do a kind of debounce of a request on the server.

Conclusion

Idempotence is a fundamental concept for developing reliable and resilient applications. By ensuring that repeated operations do not cause unexpected effects, we can design more robust and secure systems. Implementing idempotence requires planning and good practices, but the benefits in terms of consistency and reliability make this effort worthwhile.