Everything About Transactions in Java: From ACID to Distributed Systems

-- Transfer 500,000 from account A to account B
UPDATE accounts SET balance = balance - 500000 WHERE id = 'A';
UPDATE accounts SET balance = balance + 500000 WHERE id = 'B';

User 1: reads A.balance = 1,000,000        |
                                            |
User 2: reads A.balance = 1,000,000        |
                                            |
User 1: writes A.balance = 500,000         |  <- deducts 500k
                                            |
User 2: writes A.balance = 500,000         |  <- deducts another 500k but read from 1,000,000!
                                            |
Actual: A.balance = 500,000                |  (should be 0 if both transfers were valid)
        or A.balance = -500,000            |  (if no check exists)

// Without transaction -- real code from a production legacy system:
public void placeOrder(OrderRequest request) {
    // Step 1: Create order
    long orderId = orderDao.insert(request);

    // Step 2: Deduct inventory
    for (OrderItem item : request.getItems()) {
        inventoryDao.deduct(item.getProductId(), item.getQuantity());
        // If the third deduct fails (out of stock) -> order created, 2 items already deducted
        // Database is in an inconsistent state
    }

    // Step 3: Create payment record
    paymentDao.createPending(orderId, request.getTotalAmount());
    // If this fails -> order created, inventory deducted, no payment record
}

Transaction running -> crash -> restart
PostgreSQL reads WAL
If it sees BEGIN but no COMMIT -> undo all changes from that transaction
If it sees COMMIT -> redo if data was not yet flushed to disk

WAL entries:
[BEGIN txn=1234]
[UPDATE accounts SET balance=500000 WHERE id='A', old_val=1000000]
[UPDATE accounts SET balance=1500000 WHERE id='B', old_val=1000000]
[COMMIT txn=1234]

Crash after COMMIT entry -> recovery redoes both UPDATEs
Crash before COMMIT entry -> recovery undoes both UPDATEs

-- Database "consistent" but business logic is wrong:
BEGIN;
UPDATE accounts SET balance = -500000 WHERE id = 'A'; -- Valid if no CHECK constraint
COMMIT;
-- Database is OK (no constraint violated), but a negative balance is wrong for business

Row "account A" in MVCC:
Version 1: balance=1,000,000 (created_by=txn_100, deleted_by=txn_200)
Version 2: balance=500,000   (created_by=txn_200, deleted_by=null)

Transaction txn_300 starts while txn_200 is running:
-> txn_300 sees Version 1 (txn_200 has not committed yet)
-> txn_300 reads balance=1,000,000

Transaction txn_400 starts after txn_200 commits:
-> txn_400 sees Version 2
-> txn_400 reads balance=500,000

Application          PostgreSQL
    |                    |
    |-- BEGIN ---------->|  PostgreSQL assigns a transaction ID (XID)
    |                    |  Snapshot created (depending on isolation level)
    |                    |
    |-- UPDATE --------->|  Change written to shared buffer (memory)
    |                    |  WAL entry written to WAL buffer
    |                    |  Lock acquired on affected rows
    |                    |
    |-- SELECT --------->|  Read from snapshot, uncommitted changes not visible
    |                    |
    |-- COMMIT --------->|  WAL buffer flushed to disk (fsync)
    |                    |  Locks released
    |                    |  XID marked as committed in pg_xact
    |<-- OK -------------|
    |                    |
    |                    |  [Background] VACUUM cleans up old versions

Time    T1 (User 1 votes)         T2 (User 2 votes)         DB
 1      READ votes = 100
 2                                READ votes = 100
 3      WRITE votes = 101
 4                                WRITE votes = 101          <- T1's update lost!
 5                                COMMIT

Result: votes = 101 (should be 102)

-- Reproduce in PostgreSQL (isolation: READ COMMITTED)
-- Session 1:
BEGIN;
SELECT vote_count FROM posts WHERE id = 1; -- 100
-- (pause)
UPDATE posts SET vote_count = 101 WHERE id = 1;
COMMIT;

-- Session 2 (running concurrently):
BEGIN;
SELECT vote_count FROM posts WHERE id = 1; -- 100
UPDATE posts SET vote_count = 101 WHERE id = 1; -- Overwrites Session 1!
COMMIT;

-- Atomic update -- no race condition
UPDATE posts SET vote_count = vote_count + 1 WHERE id = 1;

Time    T1 (Payment processing)    T2 (Fraud check)          DB
 1      BEGIN
 2      UPDATE payment SET status='PROCESSING'
 3                                 BEGIN
 4                                 READ payment status = 'PROCESSING'  <- Dirty read!
 5                                 (decides: not fraud, allow)
 6      -- payment gateway fails
 7      ROLLBACK
 8                                 (fraud check based on data that never existed)

Time    T1 (Report generation)     T2 (Order update)         DB
 1      BEGIN
 2      SELECT total FROM orders WHERE id=1  -> 500k
 3                                 BEGIN
 4                                 UPDATE orders SET total=600k WHERE id=1
 5                                 COMMIT
 6      SELECT total FROM orders WHERE id=1  -> 600k  <- Different from first read!
 7      COMMIT

T1 sees two different values for the same row within the same transaction.

Time    T1 (Inventory check)       T2 (New reservation)      DB
 1      BEGIN
 2      SELECT COUNT(*) FROM reservations
         WHERE product_id=5 AND date='2026-06-07'  -> 9 (max is 10)
 3                                 BEGIN
 4                                 INSERT INTO reservations (product_id, date)
                                    VALUES (5, '2026-06-07')
 5                                 COMMIT
 6      SELECT COUNT(*) FROM reservations
         WHERE product_id=5 AND date='2026-06-07'  -> 10  <- Phantom row!
 7      (T1 sees a slot available -> books -> slot was already full)
 8      INSERT INTO reservations... <- Overbooking!

// Spring Boot -- default isolation (READ COMMITTED)
@Transactional
public ReportResult generateReport(Long merchantId) {
    // Statement 1 takes a snapshot at t=100
    long orderCount = orderRepo.countByMerchant(merchantId);

    // Between the two queries, another transaction commits more orders

    // Statement 2 takes a new snapshot at t=105
    BigDecimal totalRevenue = orderRepo.sumRevenueByMerchant(merchantId);

    // orderCount and totalRevenue may be inconsistent with each other!
    return new ReportResult(orderCount, totalRevenue);
}

@Transactional(isolation = Isolation.REPEATABLE_READ)
public ReportResult generateReport(Long merchantId) {
    // Both queries use the snapshot taken at BEGIN
    long orderCount = orderRepo.countByMerchant(merchantId);   // consistent
    BigDecimal totalRevenue = orderRepo.sumRevenueByMerchant(merchantId); // consistent
    // Guaranteed: orderCount and totalRevenue are consistent with each other
    return new ReportResult(orderCount, totalRevenue);
}

@Transactional(isolation = Isolation.SERIALIZABLE)
public void bookLastTicket(Long eventId, Long userId) {
    int available = ticketRepo.countAvailable(eventId);
    if (available > 0) {
        ticketRepo.createBooking(eventId, userId);
        ticketRepo.decrementAvailable(eventId);
    }
}
// If two users call this simultaneously:
// PostgreSQL detects a serialization conflict -> aborts one transaction
// Application receives SerializationFailureException -> must retry

// No setting -> uses database default (PostgreSQL: READ COMMITTED)
@Transactional
public void defaultMethod() { ... }

// Explicit
@Transactional(isolation = Isolation.REPEATABLE_READ)
public void consistentReport() { ... }

// Quarkus -- similar approach
@Transactional
@io.quarkus.narayana.jta.QuarkusTransaction
public void quarkusMethod() { ... }
// Isolation level via JPA properties or raw JDBC

// Before Spring -- JDBC boilerplate:
Connection conn = dataSource.getConnection();
conn.setAutoCommit(false);
try {
    // business logic
    conn.commit();
} catch (Exception e) {
    conn.rollback();
    throw e;
} finally {
    conn.setAutoCommit(true);
    conn.close();
}

@Service
public class OrderService {
    @Transactional
    public void placeOrder(OrderRequest request) {
        // Business logic
    }
}

// When Spring starts, it creates:
class OrderService$$SpringProxy extends OrderService {
    @Override
    public void placeOrder(OrderRequest request) {
        TransactionStatus status = transactionManager.getTransaction(txDef);
        try {
            super.placeOrder(request); // calls the real method
            transactionManager.commit(status);
        } catch (RuntimeException e) {
            transactionManager.rollback(status);
            throw e;
        }
    }
}

// The injected bean is the proxy, not OrderService directly:
@Autowired
private OrderService orderService; // Actually OrderService$$SpringProxy

Component A        Spring Proxy                OrderService
    |                   |                           |
    |-- placeOrder() -->|                           |
    |                   |-- getTransaction() ------>| (TransactionManager)
    |                   |<-- TransactionStatus -----|
    |                   |                           |
    |                   |-- super.placeOrder() ---->|
    |                   |                           |-- (SQL operations)
    |                   |<-- return ----------------|
    |                   |                           |
    |                   |-- commit() -------------->|
    |<-- return --------|                           |

// DataSourceTransactionManager -- used with plain JDBC
@Bean
public PlatformTransactionManager transactionManager(DataSource dataSource) {
    return new DataSourceTransactionManager(dataSource);
}

// JpaTransactionManager -- used with Hibernate/JPA
// Spring Boot auto-configures this if JPA dependency is present
@Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory emf) {
    JpaTransactionManager tm = new JpaTransactionManager(emf);
    tm.setEntityManagerFactory(emf);
    return tm;
}
// JpaTransactionManager synchronizes the JPA EntityManager with the JDBC transaction
// -> Both JPA and JDBC operations in the same transaction use the same Connection

@Service
public class OrderService {
    @Transactional
    public void placeOrder(OrderRequest request) {
        createOrder(request);
        reserveInventory(request);
    }

    @Transactional(propagation = Propagation.REQUIRES_NEW)
    public void reserveInventory(OrderRequest request) {
        // Intended to run in a separate transaction for independent logging
        // BUT: called from the same class -> bypasses the proxy -> no new transaction!
        inventoryRepo.reserve(request);
        auditLog.record("RESERVED", request);
    }
}

// Why? Because this.reserveInventory() calls the method directly on the object,
// not through the Spring proxy. The proxy only intercepts calls from outside the class.

@Service
public class OrderService {
    @Autowired
    private OrderService self; // inject the proxy of this bean

    @Transactional
    public void placeOrder(OrderRequest request) {
        createOrder(request);
        self.reserveInventory(request); // call through proxy -> REQUIRES_NEW takes effect
    }

    @Transactional(propagation = Propagation.REQUIRES_NEW)
    public void reserveInventory(OrderRequest request) {
        inventoryRepo.reserve(request);
    }
}

@Service
public class OrderService {
    @Autowired
    private InventoryService inventoryService;

    @Transactional
    public void placeOrder(OrderRequest request) {
        createOrder(request);
        inventoryService.reserveInventory(request); // different bean -> goes through proxy
    }
}

@Service
public class InventoryService {
    @Transactional(propagation = Propagation.REQUIRES_NEW)
    public void reserveInventory(OrderRequest request) {
        inventoryRepo.reserve(request);
    }
}

@Service
public class OrderService {
    @Transactional // <- HAS NO EFFECT on private methods
    private void createOrderInternal(OrderRequest request) {
        // Spring proxy cannot override private methods
        // -> no transaction
    }

    public void placeOrder(OrderRequest request) {
        createOrderInternal(request); // Called directly, not through proxy
    }
}

@Service
public class PaymentService {
    @Transactional
    public void processPayment(PaymentRequest request) throws PaymentException {
        chargeCard(request);
        updateBalance(request);
        // If chargeCard throws PaymentException (checked) -> NO rollback!
        // Spring defaults to rolling back only for RuntimeException and Error
    }
}

// Fix:
@Transactional(rollbackFor = PaymentException.class)
public void processPayment(PaymentRequest request) throws PaymentException {
    chargeCard(request);
    updateBalance(request);
}

// Or: rollback for all Exception types
@Transactional(rollbackFor = Exception.class)
public void processPayment(PaymentRequest request) throws Exception { ... }

@Service
public class NotificationService {
    @Async
    @Transactional
    public void sendConfirmationEmail(Long orderId) {
        Order order = orderRepo.findById(orderId).orElseThrow();
        // order.getItems() -> may throw LazyInitializationException
        // because the method runs on a different thread with no HTTP request context,
        // and OSIV will not work correctly here
        emailService.send(buildEmail(order));
    }
}

// @Async runs on a separate thread pool -> transaction context is not propagated
// The new thread has no TransactionSynchronizationManager binding
// -> @Transactional is ignored or creates an unrelated new transaction

@Service
public class OrderService {
    @Transactional
    public void confirmOrder(Long orderId) {
        Order order = orderRepo.findById(orderId)
            .orElseThrow();
        order.confirm();
        OrderConfirmedDto dto = OrderConfirmedDto.from(order); // Fetch all needed data
        notificationService.sendConfirmationEmail(dto); // Pass DTO, not entity
    }
}

@Service
public class NotificationService {
    @Async // No @Transactional needed -- no DB operations
    public void sendConfirmationEmail(OrderConfirmedDto dto) {
        emailService.send(buildEmail(dto));
    }
}

@Transactional // long transaction = connection and locks held for a long time
public OrderResult processLargeOrder(LargeOrderRequest request) {
    // Step 1: Validate (fast, ~5ms)
    validateRequest(request);

    // Step 2: Call external pricing API (slow, ~200ms)
    PricingResult pricing = externalPricingService.calculate(request); // External call!

    // Step 3: Call inventory API (slow, ~150ms)
    InventoryResult inventory = externalInventoryService.check(request); // External call!

    // Step 4: Database operations (fast, ~10ms)
    Order order = createOrder(request, pricing, inventory);

    // Total transaction time: ~365ms
    // Connection held: 365ms
    // Locks held: 365ms
    // HikariCP pool exhaustion under concurrent requests!
}

public OrderResult processLargeOrder(LargeOrderRequest request) {
    validateRequest(request);

    // External calls OUTSIDE the transaction
    PricingResult pricing = externalPricingService.calculate(request); // 200ms, no lock held
    InventoryResult inventory = externalInventoryService.check(request); // 150ms, no lock held

    // Only DB operations inside the transaction
    return createOrderTransactionally(request, pricing, inventory); // ~10ms
}

@Transactional
private OrderResult createOrderTransactionally(LargeOrderRequest request,
                                                PricingResult pricing,
                                                InventoryResult inventory) {
    return createOrder(request, pricing, inventory);
}

@Service
public class OrderService {
    @Transactional
    public void placeOrder(OrderRequest request) {
        Order order = createOrder(request);
        try {
            auditService.logOrderCreated(order.getId()); // REQUIRES_NEW
        } catch (Exception e) {
            log.warn("Audit logging failed, continuing...");
            // Assumes audit failure does not affect the order
        }
        // Order still commits because audit runs in a separate transaction
        // This is CORRECT when using REQUIRES_NEW
    }
}

@Service
public class AuditService {
    @Transactional(propagation = Propagation.REQUIRED) // <- BUG: REQUIRED instead of REQUIRES_NEW
    public void logOrderCreated(Long orderId) {
        auditRepo.save(new AuditLog(orderId, "ORDER_CREATED"));
        // If this rolls back -> it rolls back the caller's order transaction too!
    }
}
// With REQUIRED: auditService runs in the SAME transaction as orderService
// If auditService throws and the caller catches -> the transaction is still marked "rollback-only"
// The whole transaction rolls back on commit

// Fix: REQUIRES_NEW for audit, because audit failure should not roll back the business operation
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void logOrderCreated(Long orderId) {
    auditRepo.save(new AuditLog(orderId, "ORDER_CREATED"));
}

@Transactional(propagation = Propagation.REQUIRED) // default
public void methodA() {
    methodB(); // methodB joins methodA's transaction
}

@Transactional(propagation = Propagation.REQUIRED)
public void methodB() {
    // If methodA has a transaction -> methodB uses the same one
    // If there is no transaction -> methodB creates a new one
}

Caller has a transaction:    Caller has no transaction:
[---methodA txn---]          [---methodB txn (new)---]
  [methodB joins]              methodB

@Transactional
public void methodA() {
    try {
        methodB(); // methodB throws and triggers rollback
    } catch (Exception e) {
        // Assumes the exception was handled and methodA continues
    }
    orderRepo.save(order); // <- throws UnexpectedRollbackException!
    // The transaction was already marked rollback-only by methodB
}

@Transactional(propagation = Propagation.REQUIRES_NEW)
public void auditLog(String action) {
    // Suspends the caller's transaction if present
    // Creates a new, completely independent transaction
    // Commit or rollback does not affect the caller's transaction
    auditRepo.save(new AuditEntry(action));
}

[---caller txn (suspended)---]
    [---new txn---]
    [---commit or rollback---]
[---caller txn (resumed)------]

@Transactional(propagation = Propagation.NESTED)
public void nestedOperation() {
    // Creates a SAVEPOINT inside the parent transaction
    // Rollback returns only to the SAVEPOINT, not the full transaction
    // Commit only happens when the parent transaction commits
}

[---outer txn---]
    SAVEPOINT sp1
    [---nested operation---]
    RELEASE SAVEPOINT sp1 (if successful)
    or ROLLBACK TO SAVEPOINT sp1 (if failed)
[---outer txn commits or rolls back entirely---]

// SUPPORTS: Use a transaction if one exists, do not create one if not
@Transactional(propagation = Propagation.SUPPORTS)
public List<Order> readOrders() {
    // Suitable for read operations that do not need a transaction
    // but benefit from one if available
}

// NOT_SUPPORTED: Suspend the transaction if one exists, run without one
@Transactional(propagation = Propagation.NOT_SUPPORTED)
public void longRunningRead() {
    // Suspends the caller's transaction to avoid holding the connection
    // No transaction needed for a pure read
}

// MANDATORY: A transaction from the caller is required, throws if absent
@Transactional(propagation = Propagation.MANDATORY)
public void criticalDbWrite() {
    // Ensures callers do not call this method outside a transaction
    // Fail fast instead of data corruption
}

// NEVER: Must not have a transaction, throws if one is present
@Transactional(propagation = Propagation.NEVER)
public void cacheLoad() {
    // Ensures this operation is not wrapped in a transaction
    // Example: cache warming does not need (and should not have) transaction overhead
}

@Transactional
public void updateOrder(Long orderId) {
    // Load entity -> Hibernate stores a reference in the persistence context
    Order order = orderRepo.findById(orderId).orElseThrow();
    // Hibernate also stores a snapshot of the initial state

    // Modify the entity
    order.setStatus("PROCESSING");
    order.setProcessedAt(Instant.now());

    // No save() needed -- Hibernate detects changes automatically on flush
    // (dirty checking at flush time)
}
// When the @Transactional method returns -> flush -> commit
// Hibernate compares current state vs snapshot -> generates UPDATE SQL
// -> UPDATE orders SET status='PROCESSING', processed_at=... WHERE id=?

[New / Transient]
     | entityManager.persist(entity)
     v
[Managed / Persistent]  <--- entityManager.find() / JPQL query
     |                         (inside an active session)
     | entityManager.detach(entity)
     | session closes
     v
[Detached]
     | entityManager.merge(detachedEntity)
     v
[Managed / Persistent]
     | entityManager.remove(entity)
     v
[Removed]
     | flush / commit
     v
[Deleted from DB]

@Transactional
public void demonstrateStates(Long orderId) {
    // MANAGED state -- Hibernate tracks all changes
    Order managed = em.find(Order.class, orderId);
    managed.setStatus("UPDATED"); // Will generate UPDATE on flush

    // DETACHED -- Hibernate does not track
    em.detach(managed);
    managed.setStatus("DETACHED_CHANGE"); // No SQL generated

    // MERGE -- Hibernate reloads and applies changes
    Order reattached = em.merge(managed);
    // Hibernate executes: SELECT * FROM orders WHERE id=?
    // Then applies state from the managed object -> generates UPDATE
}

@Transactional
public void bugScenario() {
    Order order = new Order("PENDING");
    orderRepo.save(order); // SQL has NOT run yet! Only queued in the persistence context

    // If you query immediately using native SQL:
    int count = jdbcTemplate.queryForObject(
        "SELECT COUNT(*) FROM orders WHERE status='PENDING'", Integer.class);
    // count may be 0 because the INSERT has not been flushed to the DB yet!
}

// Manual flush for precise control:
@Transactional
public void batchInsert(List<OrderData> data) {
    for (int i = 0; i < data.size(); i++) {
        Order order = new Order(data.get(i));
        em.persist(order);

        if (i % 50 == 0) {
            em.flush();  // Flush batch to DB
            em.clear();  // Clear persistence context to avoid memory bloat
        }
    }
}

@Transactional
public void loadManyEntities() {
    // Load 10,000 orders into the persistence context
    List<Order> orders = orderRepo.findAll();
    // Hibernate stores 10,000 snapshots

    // ... some logic that does not modify orders ...

    // On flush: Hibernate compares 10,000 current states vs 10,000 snapshots
    // Even if nothing changed, the overhead still occurs
}

// Fix: @Immutable or StatelessSession for read-only bulk operations
@Entity
@org.hibernate.annotations.Immutable
public class ProductCatalog {
    // Never modified -> Hibernate skips dirty checking
}

@Entity
public class Account {
    @Id
    private Long id;

    private BigDecimal balance;

    @Version // Hibernate automatically increments on update
    private Long version;
}

// Thread 1 and Thread 2 both read the same account (version=5)
// Thread 1 updates first -> version becomes 6
// Thread 2 tries to update with WHERE id=? AND version=5 -> 0 rows affected
// -> OptimisticLockException

-- SQL generated by Hibernate:
UPDATE accounts
SET balance = 900000, version = 6
WHERE id = 123 AND version = 5; -- <- Optimistic check
-- If 0 rows affected -> OptimisticLockException

@Service
public class AccountService {
    @Transactional
    @Retryable(
        retryFor = OptimisticLockingFailureException.class,
        maxAttempts = 3,
        backoff = @Backoff(delay = 100, multiplier = 2)
    )
    public void transferFunds(Long fromId, Long toId, BigDecimal amount) {
        Account from = accountRepo.findById(fromId).orElseThrow();
        Account to = accountRepo.findById(toId).orElseThrow();
        from.deduct(amount);
        to.add(amount);
    }
    // On conflict -> retry up to 3 times with exponential backoff
}

// PESSIMISTIC_WRITE: SELECT ... FOR UPDATE
@Lock(LockModeType.PESSIMISTIC_WRITE)
@Query("SELECT a FROM Account a WHERE a.id = :id")
Account findByIdForUpdate(@Param("id") Long id);

@Transactional
public void transferFunds(Long fromId, Long toId, BigDecimal amount) {
    // Lock both accounts immediately on read
    // Guarantees nobody else can modify them while we process
    Long firstId = Math.min(fromId, toId);   // Consistent lock ordering!
    Long secondId = Math.max(fromId, toId);  // Prevents deadlock

    Account first = accountRepo.findByIdForUpdate(firstId);
    Account second = accountRepo.findByIdForUpdate(secondId);

    if (firstId.equals(fromId)) {
        first.deduct(amount);
        second.add(amount);
    } else {
        second.deduct(amount);
        first.add(amount);
    }
}

-- SQL generated:
SELECT * FROM accounts WHERE id = ? FOR UPDATE;
-- Row is locked until the transaction commits or rolls back
-- Concurrent transactions block at this statement

// PESSIMISTIC_READ: SELECT ... FOR SHARE
// Allows concurrent reads, blocks writes
// Use when you need to ensure data does not change while you read it for calculation

@Lock(LockModeType.PESSIMISTIC_READ)
Account findByIdForShare(@Param("id") Long id);
// -> SELECT ... FOR SHARE (PostgreSQL)
// Multiple transactions can hold a SHARE lock simultaneously
// Only WRITE locks (FOR UPDATE) must wait

T1 locks Account A -> waits for lock on Account B
T2 locks Account B -> waits for lock on Account A
-> Deadlock! Both wait for each other forever

ERROR: deadlock detected
DETAIL: Process 1234 waits for ShareLock on transaction 5678; blocked by process 5678.
        Process 5678 waits for ShareLock on transaction 1234; blocked by process 1234.
HINT: See server log for query details.

// 1. Consistent lock ordering (shown above)
// 2. Reduce lock scope and duration

// 3. Timeout on lock acquisition
@Transactional
public void processWithTimeout(Long orderId) {
    // PostgreSQL: SET lock_timeout = '5s' before the query
    // If the lock is not acquired within 5s -> throws LockTimeoutException instead of deadlock
}

// application.properties
spring.jpa.properties.jakarta.persistence.lock.timeout=5000 // 5 seconds

Order Service    -> Order DB (PostgreSQL)
Payment Service  -> Payment DB (PostgreSQL)
Inventory Service -> Inventory DB (PostgreSQL)

// There is no mechanism to span a single transaction across all three DBs
// Each service has its own transaction and commits independently

1. Order Service: CREATE order -> commit OK
2. Payment Service: CHARGE payment -> commit OK
3. Inventory Service: DEDUCT inventory -> FAIL

State: Order exists, money charged, inventory not deducted
-> Inconsistent state with no automatic recovery

Phase 1 (Prepare):
Coordinator -> Order Service: "Prepare to commit"
Coordinator -> Payment Service: "Prepare to commit"
Coordinator -> Inventory Service: "Prepare to commit"

If all return "Ready":

Phase 2 (Commit):
Coordinator -> all: "Commit now"

Saga: Place Order

Step 1: Order Service     -> CREATE order (local tx)
Step 2: Inventory Service -> RESERVE inventory (local tx)
Step 3: Payment Service   -> CHARGE payment (local tx)
Step 4: Shipping Service  -> SCHEDULE shipment (local tx)

If Payment FAILS:
Compensation Step 3: Payment Service   -> VOID charge
Compensation Step 2: Inventory Service -> RELEASE reservation
Compensation Step 1: Order Service     -> CANCEL order

Order Service: CREATE order -> publish OrderCreated event
    |
Inventory Service: consume OrderCreated -> RESERVE -> publish InventoryReserved
    |
Payment Service: consume InventoryReserved -> CHARGE -> publish PaymentCharged
    |
Shipping Service: consume PaymentCharged -> SCHEDULE shipment

Failure:
Payment Service: CHARGE fails -> publish PaymentFailed
    |
Inventory Service: consume PaymentFailed -> RELEASE reservation -> publish InventoryReleased
    |
Order Service: consume InventoryReleased -> CANCEL order

// Choreography with Spring Events (in-process) or Kafka (cross-service)
@Service
public class PaymentService {
    @Transactional
    @KafkaListener(topics = "inventory.reserved")
    public void handleInventoryReserved(InventoryReservedEvent event) {
        try {
            PaymentResult result = chargePayment(event.getOrderId(), event.getAmount());
            kafkaTemplate.send("payment.charged",
                new PaymentChargedEvent(event.getOrderId(), result.getTransactionId()));
        } catch (PaymentException e) {
            kafkaTemplate.send("payment.failed",
                new PaymentFailedEvent(event.getOrderId(), e.getReason()));
        }
    }
}

Order Orchestrator:
  Step 1: call Order Service     -> CREATE
  Step 2: call Inventory Service -> RESERVE
  Step 3: call Payment Service   -> CHARGE
  Step 4: call Shipping Service  -> SCHEDULE

  If Step 3 fails:
    call Inventory Service -> RELEASE (compensation)
    call Order Service     -> CANCEL (compensation)

// Orchestration with Spring State Machine or Temporal
@Component
public class PlaceOrderSaga {
    @Autowired private OrderClient orderClient;
    @Autowired private InventoryClient inventoryClient;
    @Autowired private PaymentClient paymentClient;

    @Transactional
    public void execute(PlaceOrderCommand command) {
        SagaState state = sagaStateRepo.create(command);

        try {
            // Step 1
            state.setOrderId(orderClient.createOrder(command));
            sagaStateRepo.save(state);

            // Step 2
            state.setReservationId(inventoryClient.reserve(state.getOrderId(), command.getItems()));
            sagaStateRepo.save(state);

            // Step 3
            state.setPaymentId(paymentClient.charge(state.getOrderId(), command.getAmount()));
            sagaStateRepo.save(state);

            state.markCompleted();
            sagaStateRepo.save(state);

        } catch (Exception e) {
            compensate(state);
            throw e;
        }
    }

    private void compensate(SagaState state) {
        if (state.getPaymentId() != null) {
            paymentClient.void_(state.getPaymentId());
        }
        if (state.getReservationId() != null) {
            inventoryClient.release(state.getReservationId());
        }
        if (state.getOrderId() != null) {
            orderClient.cancel(state.getOrderId());
        }
    }
}

// Each step must be idempotent -- safe to retry
@Transactional
@KafkaListener(topics = "payment.charge.requested")
public void chargePayment(ChargePaymentCommand command) {
    // Check idempotency key before processing
    if (paymentRepo.existsByIdempotencyKey(command.getIdempotencyKey())) {
        // Already processed, return existing result
        return;
    }

    Payment payment = new Payment(command);
    payment.setIdempotencyKey(command.getIdempotencyKey());
    paymentRepo.save(payment);
    // Process payment...
}

@Transactional
public void placeOrder(OrderRequest request) {
    Order order = createOrder(request); // DB write
    orderRepo.save(order);
    // COMMIT (DB OK)

    // Publish message OUTSIDE the transaction
    kafkaTemplate.send("order.placed", new OrderPlacedEvent(order.getId()));
    // If Kafka publish fails -> DB committed, message not sent
    // Order exists but downstream services do not know about it
}

// Publish first, commit after:
kafkaTemplate.send("order.placed", event);
// Kafka OK
// DB commit fails -> Order does not exist, but message was published
// Downstream services process an order that does not exist

-- Schema
CREATE TABLE outbox_messages (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    aggregate_type VARCHAR(100) NOT NULL,  -- 'ORDER'
    aggregate_id VARCHAR(100) NOT NULL,    -- order ID
    event_type VARCHAR(100) NOT NULL,      -- 'ORDER_PLACED'
    payload JSONB NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    published_at TIMESTAMP WITH TIME ZONE, -- NULL = not yet published
    retry_count INT DEFAULT 0
);

CREATE INDEX idx_outbox_unpublished ON outbox_messages (created_at)
WHERE published_at IS NULL;

@Transactional
public void placeOrder(OrderRequest request) {
    Order order = createOrder(request);
    orderRepo.save(order);

    // Write outbox message in the SAME transaction
    OutboxMessage message = OutboxMessage.builder()
        .aggregateType("ORDER")
        .aggregateId(order.getId().toString())
        .eventType("ORDER_PLACED")
        .payload(objectMapper.writeValueAsString(new OrderPlacedEvent(order)))
        .build();
    outboxRepo.save(message);
    // Both order and outbox message are committed atomically
    // Or both roll back -- no dual write problem
}

// Outbox poller -- runs as a separate component
@Component
public class OutboxPoller {
    @Scheduled(fixedDelay = 1000) // Every second
    @Transactional
    public void pollAndPublish() {
        List<OutboxMessage> pending = outboxRepo.findUnpublished(50);

        for (OutboxMessage msg : pending) {
            try {
                kafkaTemplate.send(
                    topicFor(msg.getEventType()),
                    msg.getAggregateId(),
                    msg.getPayload()
                ).get(5, TimeUnit.SECONDS); // Synchronous send with timeout

                msg.markPublished();
                outboxRepo.save(msg);
            } catch (Exception e) {
                msg.incrementRetry();
                outboxRepo.save(msg);
                log.error("Failed to publish outbox message {}", msg.getId(), e);
            }
        }
    }
}

# Debezium connector configuration
{
  "name": "outbox-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "postgres",
    "database.dbname": "orders",
    "table.include.list": "public.outbox_messages",
    "transforms": "outbox",
    "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter"
  }
}

-- View all active locks and who is blocking whom
SELECT
    blocked.pid AS blocked_pid,
    blocked.usename AS blocked_user,
    blocked.query AS blocked_query,
    now() - blocked.query_start AS blocked_duration,
    blocking.pid AS blocking_pid,
    blocking.query AS blocking_query,
    blocking.state AS blocking_state
FROM pg_stat_activity blocked
JOIN pg_stat_activity blocking
    ON blocking.pid = ANY(pg_blocking_pids(blocked.pid))
ORDER BY blocked_duration DESC;

-- View long-running transactions (longer than 1 minute)
SELECT
    pid,
    now() - xact_start AS txn_duration,
    now() - query_start AS query_duration,
    state,
    wait_event_type,
    wait_event,
    left(query, 100) AS query_snippet
FROM pg_stat_activity
WHERE xact_start IS NOT NULL
  AND now() - xact_start > INTERVAL '1 minute'
ORDER BY txn_duration DESC;

-- Kill a long-running transaction (emergency only)
SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE now() - xact_start > INTERVAL '10 minutes'
  AND state != 'idle';

// HikariCP metrics via Micrometer
@Bean
public MeterRegistryCustomizer<MeterRegistry> hikariMetrics(DataSource dataSource) {
    return registry -> {
        if (dataSource instanceof HikariDataSource ds) {
            ds.setMetricRegistry(registry);
        }
    };
}

// Metrics:
// hikaricp.connections.active   -> connections currently in use
// hikaricp.connections.pending  -> requests waiting for a connection (CRITICAL if > 0)
// hikaricp.connections.timeout  -> count of connection acquisition timeouts
// hikaricp.connections.acquire  -> histogram, time to acquire a connection

// Alert when pending > 5 for 30 seconds -> pool is exhausted

// Enable leak detection
# application.yml
spring:
  datasource:
    hikari:
      leak-detection-threshold: 30000  # 30 seconds without release -> log warning

// Warning: generate_statistics has overhead in production
// Enable only when debugging, disable afterward

@Scheduled(fixedRate = 60000) // Every minute
public void logHibernateStats() {
    Statistics stats = sessionFactory.getStatistics();
    long queryCount = stats.getQueryExecutionCount();
    long slowQueries = stats.getQueryExecutionMaxTime();

    if (slowQueries > 1000) { // > 1 second
        log.warn("Slow query detected: {}ms, SQL: {}",
            slowQueries, stats.getQueryExecutionMaxTimeQueryString());
    }

    // Reset each interval to calculate rates
    stats.clear();
}

@Aspect
@Component
public class TransactionMonitoringAspect {
    private final MeterRegistry meterRegistry;

    @Around("@annotation(transactional)")
    public Object monitorTransaction(ProceedingJoinPoint pjp,
                                      Transactional transactional) throws Throwable {
        Timer.Sample sample = Timer.start(meterRegistry);
        String methodName = pjp.getSignature().toShortString();
        try {
            Object result = pjp.proceed();
            sample.stop(Timer.builder("transaction.duration")
                .tag("method", methodName)
                .tag("outcome", "success")
                .register(meterRegistry));
            return result;
        } catch (Exception e) {
            sample.stop(Timer.builder("transaction.duration")
                .tag("method", methodName)
                .tag("outcome", "failure")
                .register(meterRegistry));
            throw e;
        }
    }
}

# Transaction duration P99 > 5 seconds
- alert: SlowTransactions
  expr: histogram_quantile(0.99, rate(transaction_duration_seconds_bucket[5m])) > 5
  for: 2m
  labels:
    severity: warning

# Connection pool pending > 0
- alert: ConnectionPoolPressure
  expr: hikaricp_connections_pending > 0
  for: 30s
  labels:
    severity: critical

-- Find duplicate payments within a short time window
SELECT
    account_id,
    amount,
    COUNT(*) as count,
    MIN(created_at) as first,
    MAX(created_at) as last
FROM payment_transactions
WHERE created_at > NOW() - INTERVAL '1 hour'
GROUP BY account_id, amount
HAVING COUNT(*) > 1;

User clicks "Pay" -> Request 1 sent
Network timeout (30s) -> User sees an error -> Clicks again
Request 1 is still processing (slow because of external payment gateway)
Request 2 is also processed
Both succeed -> Two payments

@Transactional
public PaymentResult processPayment(PaymentRequest request) {
    // Check idempotency key BEFORE processing
    Optional<Payment> existing = paymentRepo.findByIdempotencyKey(
        request.getIdempotencyKey()
    );
    if (existing.isPresent()) {
        return PaymentResult.from(existing.get()); // Return previous result
    }

    Payment payment = createPayment(request);
    payment.setIdempotencyKey(request.getIdempotencyKey());
    paymentRepo.save(payment);
    return PaymentResult.from(payment);
}

// Database constraint enforces uniqueness even under race conditions:
// CREATE UNIQUE INDEX idx_payments_idempotency ON payments (idempotency_key);

// Broken code:
@Transactional
public void reserveInventory(Long productId, int quantity) {
    Product product = productRepo.findById(productId).orElseThrow();
    // 100 concurrent requests all read stock = 100
    if (product.getStock() < quantity) {
        throw new InsufficientStockException();
    }
    product.setStock(product.getStock() - quantity); // Lost update!
    // Each request sets stock = 100 - 1 = 99, but concurrent reads do not see each other's writes
    productRepo.save(product);
}

@Transactional
public void reserveInventory(Long productId, int quantity) {
    // Atomic UPDATE with check -- no prior read needed
    int updated = productRepo.decrementStockIfAvailable(productId, quantity);
    if (updated == 0) {
        throw new InsufficientStockException();
    }
}

// Repository:
@Modifying
@Query("""
    UPDATE Product p
    SET p.stock = p.stock - :quantity
    WHERE p.id = :productId AND p.stock >= :quantity
    """)
int decrementStockIfAvailable(@Param("productId") Long productId,
                               @Param("quantity") int quantity);
// Atomic: check and update in a single SQL statement
// Database-level atomicity eliminates the race condition

@Transactional
public void placeOrder(OrderRequest request) throws PaymentException {
    Order order = createAndSaveOrder(request);
    // ORDER is saved inside the transaction

    try {
        paymentService.charge(order.getId(), request.getAmount());
        // PaymentException (checked) is caught here
    } catch (PaymentException e) {
        log.error("Payment failed for order {}", order.getId(), e);
        // Developer assumed the transaction would roll back
        // BUT: checked exceptions do NOT trigger rollback by default!
        throw e; // Rethrown, but the transaction did not roll back
    }
}
// Result: ORDER commits successfully, payment fails -> inconsistent state

@Transactional(rollbackFor = PaymentException.class) // Explicit rollback
public void placeOrder(OrderRequest request) throws PaymentException {
    Order order = createAndSaveOrder(request);
    paymentService.charge(order.getId(), request.getAmount());
    // If PaymentException is thrown -> the entire order rolls back
}

@Transactional
public RegistrationResult registerUser(RegistrationRequest request) {
    User user = createUser(request);
    userRepo.save(user);
    sendWelcomeEmail(user); // SMTP call: 2 to 5 seconds, sometimes times out at 30 seconds!
    // Connection is held for the entire SMTP call
    return RegistrationResult.success(user.getId());
}

@Transactional
public RegistrationResult registerUser(RegistrationRequest request) {
    User user = createUser(request);
    userRepo.save(user);
    return RegistrationResult.success(user.getId());
    // Transaction commits here, connection released
}

// Call email OUTSIDE the transaction
public RegistrationResult register(RegistrationRequest request) {
    RegistrationResult result = registerUser(request); // transaction
    sendWelcomeEmail(result.getUserId()); // after transaction commits
    return result;
}

[] Does this operation actually need a transaction?
  - If it is a single SQL statement -> the DB is already atomic
  - If it is a pure read -> @Transactional(readOnly=true) or no annotation needed

[] Which isolation level is appropriate?
  - READ COMMITTED: default, most OLTP operations
  - REPEATABLE READ: consistent snapshot (reports, audits)
  - SERIALIZABLE: critical correctness (booking, financial)

[] Transaction scope -- include only:
  - Database operations
  - Exclude: external API calls, file I/O, email/SMS, heavy computation

[] Is idempotency needed?
  - If the operation may be retried -> needs an idempotency key

[] @Transactional(rollbackFor = Exception.class) when using checked exceptions
[] Propagation is chosen deliberately (do not blindly rely on REQUIRED)
[] No self-invocation (calling @Transactional methods within the same class)
[] Method is public (Spring proxy does not intercept private/package-private)
[] @Async and @Transactional are not mixed
[] Locking strategy chosen correctly:
  - Optimistic: low contention, retry acceptable
  - Pessimistic: high contention, correctness critical
[] Lock ordering is consistent when locking multiple entities (prevents deadlock)
[] readOnly=true on all read-only service methods

[] @Transactional on all methods without considering scope
[] External API calls inside a @Transactional method
[] try-catch swallowing exceptions without rethrowing -> may hide rollback behavior
[] @Transactional on a private method (silently a no-op)
[] Checked exceptions not declared in rollbackFor
[] EAGER fetching on OneToMany/ManyToMany
[] N+1 queries not detected (requires Hibernate statistics)
[] Outbox pattern missing for critical events
[] Missing idempotency key for payment/financial operations

[] spring.jpa.open-in-view=false
[] HikariCP pool metrics exposed and alerted
[] Transaction duration metrics exposed and alerted (P99 > threshold)
[] Slow query log enabled (log_min_duration_statement = 100ms)
[] Deadlock detection: pg_stat_activity monitoring
[] Connection pool sized correctly:
  pool_size = (core_count x 2) + 1 per instance
  total = pool_size x number_of_instances < DB max_connections

[] Statement timeout configured (prevents hung transactions):
  spring.jpa.properties.jakarta.persistence.query.timeout=30000

Symptom: Application timeout / slow
[] Check hikaricp_connections_pending > 0?
  -> Pool exhausted -> find long transactions
  SELECT pid, now()-xact_start, query FROM pg_stat_activity
  WHERE xact_start IS NOT NULL ORDER BY xact_start;

Symptom: High database CPU
[] Check pg_stat_statements for top CPU consumers
[] Check seq_scan in pg_stat_user_tables
[] Run EXPLAIN ANALYZE on slow queries

Symptom: Data inconsistency
[] Check transaction boundaries -- is the operation atomic?
[] Check rollbackFor -- do checked exceptions trigger rollback?
[] Check idempotency -- is there duplicate processing?
[] Check isolation level -- is a stale read causing inconsistency?

Symptom: Deadlock
[] Check pg_log for "deadlock detected"
[] Verify lock ordering is consistent
[] Increase lock_timeout and implement retry
[] Reduce transaction scope to decrease lock hold time