System Architecture Design

Overview

• What you’ll learn:
  • How to design scalable iDempiere deployments using multi-tenant architecture, clustering, and high availability patterns
  • How to implement database replication, load balancing, and disaster recovery strategies for production environments
  • How to plan capacity, monitor system health, and deploy iDempiere on cloud platforms using infrastructure as code
• Prerequisites: Lessons 1-39 (complete Beginner, Intermediate, and Advanced paths), production iDempiere administration experience
• Estimated reading time: 25 minutes

Introduction

A successful iDempiere implementation eventually faces a critical inflection point: the system must scale beyond a single server serving a handful of users. Whether you are onboarding additional business entities as tenants, expanding to new geographies, or simply preparing for growth, the architecture decisions you make at this stage will determine system reliability, performance, and operational cost for years to come.

This lesson covers the full spectrum of system architecture design for iDempiere — from multi-tenant models and clustering strategies to database replication, disaster recovery, and cloud deployment. The goal is to equip you with the knowledge to design an iDempiere deployment that meets enterprise-grade availability and scalability requirements.

Multi-Tenant Architecture Patterns

iDempiere’s built-in Client/Organization model provides native multi-tenancy at the application level. Every record in the database carries AD_Client_ID and AD_Org_ID columns, and the security framework enforces strict data isolation between clients. However, there are multiple ways to architect a multi-tenant deployment, each with distinct trade-offs.

Single Instance, Multi-Client

This is iDempiere’s native multi-tenancy model. A single application server and a single database host multiple clients. Each client has its own chart of accounts, business partners, products, and transactional data, all stored in the same database tables but isolated by AD_Client_ID.

• Advantages: Lowest infrastructure cost, simplest administration, single codebase and plugin set to maintain, shared system-level configuration (reference lists, countries, currencies).
• Disadvantages: A heavy workload in one client can affect performance for all clients (noisy neighbor problem). Database schema changes apply to all clients simultaneously. Upgrade and maintenance windows affect everyone.
• Best for: Related business entities under a single parent company, SaaS providers with small-to-medium tenants, development and staging environments.

Multiple Instances, Separate Databases

Each tenant gets its own iDempiere application server and its own database. The instances are completely independent and share nothing at the application level.

• Advantages: Complete workload isolation, independent upgrade schedules, tenant-specific customizations and plugin versions, no noisy neighbor risk, simpler backup and restore per tenant.
• Disadvantages: Higher infrastructure cost, more operational complexity (N instances to monitor, patch, and upgrade), no shared data between tenants without additional integration.
• Best for: Large enterprises with independent business units, managed service providers hosting unrelated companies, regulatory environments requiring strict data separation.

Hybrid Approach

Group related tenants into shared instances while keeping large or high-security tenants on dedicated instances. This balances cost efficiency with isolation requirements. Use resource quotas, connection pooling limits, and monitoring to manage the shared instances.

Horizontal Scaling Strategies

Vertical scaling (adding more CPU, RAM, and faster storage to a single server) has limits. Horizontal scaling distributes the workload across multiple servers. For iDempiere, horizontal scaling requires careful consideration of the application’s stateful nature.

Application-Level Scaling

iDempiere maintains user session state on the application server. This means you cannot simply place multiple iDempiere instances behind a round-robin load balancer — a user’s requests must consistently reach the same instance (session affinity/sticky sessions). There are two primary approaches:

• Sticky sessions with a load balancer: Configure your load balancer (HAProxy, Nginx, AWS ALB) to route all requests from a given session to the same backend instance. This is the simplest approach but limits failover — if an instance dies, active sessions on that instance are lost.
• Externalized session store: Store session data in a shared external store (Redis, Memcached, or a database) so that any instance can serve any request. This enables true stateless scaling but requires custom modifications to iDempiere’s session management layer, which is a non-trivial effort.

Database-Level Scaling

The database is often the bottleneck before the application server. Database scaling strategies include:

• Read replicas: Route read-heavy queries (reports, dashboards, lookups) to replica databases while writes go to the primary. This requires either application-level routing or a database proxy like PgBouncer with read/write splitting.
• Connection pooling: Use PgBouncer or PgPool-II in front of PostgreSQL to manage connection limits efficiently. iDempiere can consume many database connections, especially under high concurrency.
• Table partitioning: For very large transactional tables (C_Order, C_Invoice, Fact_Acct), PostgreSQL table partitioning by date range or client can dramatically improve query performance and maintenance operations.

Clustering iDempiere Instances

Clustering multiple iDempiere instances provides both high availability and increased capacity. There are two fundamental clustering models.

Shared-Database Clustering

Multiple iDempiere application server instances connect to the same database. Each instance handles a subset of users, and the database is the single source of truth. This is the most common clustering model for iDempiere.

• Architecture: Load balancer (with sticky sessions) in front of N iDempiere instances, all pointing to one PostgreSQL primary.
• Considerations: Database connection limits must accommodate all instances. Cache invalidation across instances must be addressed — when one instance modifies cached data (such as Application Dictionary metadata), other instances may serve stale cache until their next refresh cycle. Application-level schedulers (e.g., accounting processors, workflow processors) must run on only one instance to avoid duplicate processing.
• Scheduler coordination: Designate one instance as the scheduler leader or use a distributed lock mechanism (database advisory locks or a tool like Apache ZooKeeper) to ensure only one instance runs scheduled processes.

Shared-Nothing Clustering

Each iDempiere instance has its own database, and tenants are partitioned across instances. There is no shared state between instances at the application or database level.

• Architecture: A routing layer (DNS-based, or a reverse proxy with path-based routing) directs each tenant to its assigned instance.
• Considerations: Simpler to reason about (no cross-instance cache issues), easier to scale incrementally (add a new instance for a new tenant), but requires a management layer to handle tenant provisioning and routing.

Database Replication

Database replication is the foundation of both high availability and read scaling. PostgreSQL offers two primary replication mechanisms.

Streaming Replication (Physical Replication)

PostgreSQL streaming replication copies the entire database cluster at the WAL (Write-Ahead Log) level. The standby server is a byte-for-byte replica of the primary.

# postgresql.conf on primary
wal_level = replica
max_wal_senders = 5
wal_keep_size = '1GB'

# Enable archiving for point-in-time recovery
archive_mode = on
archive_command = 'cp %p /var/lib/postgresql/archive/%f'

# On standby, initialize with pg_basebackup
pg_basebackup -h primary-host -D /var/lib/postgresql/data -U replicator -P -R

# The -R flag creates standby.signal and sets primary_conninfo
# in postgresql.auto.conf automatically

• Synchronous vs asynchronous: Asynchronous replication has minimal performance impact on the primary but allows a small window of data loss if the primary fails. Synchronous replication guarantees zero data loss but adds latency to every write transaction (the primary waits for the standby to confirm receipt).
• Best for: High availability failover, read replicas for reporting, disaster recovery.

Logical Replication

Logical replication decodes WAL changes into logical operations (INSERT, UPDATE, DELETE) and applies them to the subscriber. Unlike streaming replication, the subscriber can have a different schema, indexes, or even be a different PostgreSQL version.

# On the publisher
CREATE PUBLICATION idempiere_pub FOR ALL TABLES;

# On the subscriber
CREATE SUBSCRIPTION idempiere_sub
    CONNECTION 'host=primary-host dbname=idempiere user=replicator'
    PUBLICATION idempiere_pub;

• Best for: Selective table replication, cross-version upgrades, data warehouse feeding, partial replicas for specific reporting needs.

Read Replicas for Reporting

Reporting workloads (JasperReports, custom SQL reports, BI tools) can be extremely resource-intensive. Directing them to a read replica prevents report execution from degrading the performance of transactional users. Implementation approaches include:

• Configure a separate JDBC connection pool in iDempiere that points to the replica, and route reporting processes to use this pool.
• Use a database proxy with read/write splitting rules.
• Configure BI tools (Metabase, Apache Superset, Pentaho) to connect directly to the replica.

Disaster Recovery Planning

Disaster recovery (DR) is not optional for production ERP systems. A comprehensive DR plan covers data backup, infrastructure redundancy, and documented recovery procedures.

Backup Strategies

A robust backup strategy includes multiple layers:

• Database backups: Use pg_dump for logical backups (portable, selective) and pg_basebackup for physical backups (faster restore for large databases). Schedule both daily. Enable WAL archiving for point-in-time recovery (PITR) — this lets you restore to any moment in time, not just the last backup.
• File system backups: Back up the iDempiere installation directory (plugins, configuration files, custom reports), attachment storage, and any external document repositories. Use incremental backup tools like rsync or cloud-native snapshots.
• Configuration backups: Export 2Pack packages of critical Application Dictionary customizations. Version-control all plugin source code, migration scripts, and deployment configurations in Git.

Recovery Time and Recovery Point Objectives

Define your targets clearly:

• RPO (Recovery Point Objective): How much data loss is acceptable? With synchronous streaming replication and WAL archiving, RPO can be near zero. With daily backups only, RPO is up to 24 hours.
• RTO (Recovery Time Objective): How quickly must the system be operational after a failure? This determines whether you need a hot standby (minutes), a warm standby (tens of minutes), or a cold restore from backup (hours).

DR Testing

A disaster recovery plan that has not been tested is not a plan — it is a hope. Schedule regular DR drills:

1. Restore a database backup to a separate server and verify data integrity.
2. Practice failover to a streaming replica and confirm the application functions correctly.
3. Time the entire recovery process and compare against your RTO target.
4. Document the exact steps and update the runbook after each drill.

High Availability Patterns

High availability (HA) eliminates single points of failure across every layer of the architecture.

Application Server HA

Deploy at least two iDempiere instances behind a load balancer. The load balancer performs health checks (HTTP health endpoint or TCP connection check) and removes unhealthy instances from the pool. Users on a failed instance will lose their session but can log in again and be routed to a healthy instance.

Database HA

Use PostgreSQL streaming replication with automatic failover. Tools like Patroni (combined with etcd or Consul) provide automated leader election, failover, and replica management. When the primary fails, Patroni promotes a replica to primary and reconfigures the remaining replicas — typically within seconds.

# Patroni configuration excerpt (patroni.yml)
scope: idempiere-cluster
name: node1

restapi:
  listen: 0.0.0.0:8008

postgresql:
  listen: 0.0.0.0:5432
  data_dir: /var/lib/postgresql/data
  parameters:
    max_connections: 200
    shared_buffers: 4GB
    wal_level: replica
    max_wal_senders: 5

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576

Load Balancer HA

The load balancer itself must not be a single point of failure. Use a pair of load balancers in active-passive or active-active mode with a virtual IP (keepalived/VRRP) or use a cloud-managed load balancer (AWS ALB/NLB, Azure Load Balancer, GCP Cloud Load Balancing) which is inherently redundant.

Storage HA

If iDempiere stores attachments or documents on the file system, use a shared storage solution (NFS with redundancy, AWS EFS, Azure Files, GlusterFS) so that all application instances access the same file store. Alternatively, configure iDempiere to store attachments in the database (the default behavior), which is then protected by database replication.

Capacity Planning

Capacity planning ensures your infrastructure can handle current and projected workloads. Key metrics to size include:

Sizing Guidelines

These are starting points. Actual requirements depend heavily on the volume of transactions, the complexity of customizations, reporting load, and the number of concurrent (not just named) users.

Performance Baselines

Establish baselines early in the deployment lifecycle. Measure and record:

• Average response time for key transactions (document completion, record saves, report generation)
• Database query execution times for critical queries
• JVM heap utilization, garbage collection frequency and pause times
• Database connection pool utilization
• Disk I/O throughput and latency

Monitoring and Alerting

Production iDempiere systems require comprehensive monitoring at every layer.

Application Monitoring

• JMX metrics: iDempiere exposes JVM metrics via JMX (Java Management Extensions). Use tools like Prometheus with JMX Exporter to collect heap usage, thread counts, GC statistics, and class loading data.
• Application logs: Centralize iDempiere logs using the ELK stack (Elasticsearch, Logstash, Kibana) or a managed log service. Monitor for error patterns, slow queries, and authentication failures.
• Health endpoints: Implement a custom health check servlet or plugin that verifies database connectivity, cache state, and scheduler status, and expose it at a known URL for load balancer health checks.

Database Monitoring

• pg_stat_statements: Enable this PostgreSQL extension to track query execution statistics. Identify the slowest and most frequently executed queries for optimization.
• Replication lag: Monitor the delay between primary and replicas. Alerting thresholds should be set based on your RPO — if lag exceeds the acceptable data loss window, trigger an alert.
• Connection counts: Alert when connection usage approaches max_connections. Connection exhaustion is a common cause of outages.

Infrastructure Monitoring

Use tools like Prometheus + Grafana, Datadog, or cloud-native monitoring (CloudWatch, Azure Monitor, GCP Cloud Monitoring) to track CPU, memory, disk, and network metrics across all servers. Set alerts for resource utilization thresholds (e.g., CPU above 80% sustained for 5 minutes, disk usage above 85%).

Cloud Deployment Considerations

Deploying iDempiere on cloud platforms offers elasticity, managed services, and global availability, but requires adaptation of traditional deployment practices.

Compute Options

• Virtual machines (EC2, Azure VMs, GCE): The most straightforward migration path. Run iDempiere on VMs just as you would on bare metal, but with the ability to resize, snapshot, and auto-scale.
• Containers (Docker + Kubernetes): Containerizing iDempiere enables consistent deployments, rolling updates, and orchestration. The iDempiere community maintains Docker images. Kubernetes can manage scaling, health checks, and self-healing. However, the stateful nature of iDempiere sessions requires careful configuration of persistent storage and session affinity.

Managed Database Services

Use cloud-managed PostgreSQL (AWS RDS, Azure Database for PostgreSQL, GCP Cloud SQL) to offload backup, patching, replication, and failover management. These services provide automated point-in-time recovery, read replicas, and high availability with minimal operational effort. Be aware of version compatibility — ensure the managed service supports the PostgreSQL version required by your iDempiere release.

Infrastructure as Code

Define your entire iDempiere infrastructure in code using tools like Terraform, AWS CloudFormation, or Pulumi. Infrastructure as code provides:

• Reproducibility: Spin up identical environments for development, staging, and production from the same templates.
• Version control: Track infrastructure changes in Git alongside application code.
• Disaster recovery: Rebuild the entire infrastructure from code in a new region or account if needed.

# Terraform example: PostgreSQL RDS instance for iDempiere
resource "aws_db_instance" "idempiere_db" {
  identifier           = "idempiere-production"
  engine               = "postgres"
  engine_version       = "15.4"
  instance_class       = "db.r6g.xlarge"
  allocated_storage    = 200
  storage_type         = "gp3"

  db_name              = "idempiere"
  username             = "adempiere"
  password             = var.db_password

  multi_az             = true
  backup_retention_period = 14

  vpc_security_group_ids = [aws_security_group.db_sg.id]
  db_subnet_group_name   = aws_db_subnet_group.db_subnets.name

  parameter_group_name = aws_db_parameter_group.idempiere_params.name

  tags = {
    Environment = "production"
    Application = "iDempiere"
  }
}

Putting It All Together: Reference Architecture

A production-ready iDempiere architecture for a medium-to-large deployment might look like this:

1. Load balancer tier: Cloud-managed load balancer (e.g., AWS ALB) with SSL termination and sticky sessions, distributing traffic across two or more iDempiere instances.
2. Application tier: Two or more iDempiere instances running on VMs or containers, each with 16 GB heap, configured with shared attachment storage (EFS/NFS) and one designated scheduler leader.
3. Database tier: Managed PostgreSQL (RDS Multi-AZ) with automatic failover, 64 GB RAM, provisioned IOPS storage, and a read replica for reporting workloads.
4. Monitoring: Prometheus + Grafana dashboards for application, database, and infrastructure metrics, with PagerDuty or Opsgenie for alerting.
5. Backup: Automated database snapshots with 14-day retention, WAL archiving to S3 for PITR, nightly file system backups, and 2Pack exports of AD customizations stored in version control.
6. DR: Cross-region read replica for database, infrastructure-as-code templates ready to deploy in the DR region, documented and tested runbook.

Summary

Designing an iDempiere architecture for production requires decisions across multiple dimensions: multi-tenancy model, scaling strategy, clustering approach, replication topology, and disaster recovery posture. The key principles are:

• Eliminate single points of failure at every layer.
• Separate read and write workloads where possible.
• Automate everything — backups, failover, monitoring, and infrastructure provisioning.
• Test your disaster recovery plan regularly.
• Right-size for today but architect for growth.

In the next lesson, we will explore integration patterns and middleware — how to connect your iDempiere deployment with external systems using EDI, message queues, and API-driven architectures.

# 主伺服器上的 postgresql.conf
wal_level = replica
max_wal_senders = 5
wal_keep_size = '1GB'

# 啟用歸檔以進行時間點復原
archive_mode = on
archive_command = 'cp %p /var/lib/postgresql/archive/%f'

# 在備援伺服器上，使用 pg_basebackup 初始化
pg_basebackup -h primary-host -D /var/lib/postgresql/data -U replicator -P -R

# -R 旗標自動建立 standby.signal 並在
# postgresql.auto.conf 中設定 primary_conninfo

# 在發布者上
CREATE PUBLICATION idempiere_pub FOR ALL TABLES;

# 在訂閱者上
CREATE SUBSCRIPTION idempiere_sub
    CONNECTION 'host=primary-host dbname=idempiere user=replicator'
    PUBLICATION idempiere_pub;

# Patroni 配置摘錄（patroni.yml）
scope: idempiere-cluster
name: node1

restapi:
  listen: 0.0.0.0:8008

postgresql:
  listen: 0.0.0.0:5432
  data_dir: /var/lib/postgresql/data
  parameters:
    max_connections: 200
    shared_buffers: 4GB
    wal_level: replica
    max_wal_senders: 5

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576

# Terraform 範例：iDempiere 的 PostgreSQL RDS 實例
resource "aws_db_instance" "idempiere_db" {
  identifier           = "idempiere-production"
  engine               = "postgres"
  engine_version       = "15.4"
  instance_class       = "db.r6g.xlarge"
  allocated_storage    = 200
  storage_type         = "gp3"

  db_name              = "idempiere"
  username             = "adempiere"
  password             = var.db_password

  multi_az             = true
  backup_retention_period = 14

  vpc_security_group_ids = [aws_security_group.db_sg.id]
  db_subnet_group_name   = aws_db_subnet_group.db_subnets.name

  parameter_group_name = aws_db_parameter_group.idempiere_params.name

  tags = {
    Environment = "production"
    Application = "iDempiere"
  }
}

# プライマリのpostgresql.conf
wal_level = replica
max_wal_senders = 5
wal_keep_size = '1GB'

# ポイントインタイムリカバリのためのアーカイブを有効化
archive_mode = on
archive_command = 'cp %p /var/lib/postgresql/archive/%f'

# スタンバイでpg_basebackupで初期化
pg_basebackup -h primary-host -D /var/lib/postgresql/data -U replicator -P -R

# -Rフラグはstandby.signalを作成し、
# postgresql.auto.confにprimary_conninfoを自動設定する

# パブリッシャー上
CREATE PUBLICATION idempiere_pub FOR ALL TABLES;

# サブスクライバー上
CREATE SUBSCRIPTION idempiere_sub
    CONNECTION 'host=primary-host dbname=idempiere user=replicator'
    PUBLICATION idempiere_pub;

# Patroni設定の抜粋（patroni.yml）
scope: idempiere-cluster
name: node1

restapi:
  listen: 0.0.0.0:8008

postgresql:
  listen: 0.0.0.0:5432
  data_dir: /var/lib/postgresql/data
  parameters:
    max_connections: 200
    shared_buffers: 4GB
    wal_level: replica
    max_wal_senders: 5

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576

# Terraform例：iDempiere用PostgreSQL RDSインスタンス
resource "aws_db_instance" "idempiere_db" {
  identifier           = "idempiere-production"
  engine               = "postgres"
  engine_version       = "15.4"
  instance_class       = "db.r6g.xlarge"
  allocated_storage    = 200
  storage_type         = "gp3"

  db_name              = "idempiere"
  username             = "adempiere"
  password             = var.db_password

  multi_az             = true
  backup_retention_period = 14

  vpc_security_group_ids = [aws_security_group.db_sg.id]
  db_subnet_group_name   = aws_db_subnet_group.db_subnets.name

  parameter_group_name = aws_db_parameter_group.idempiere_params.name

  tags = {
    Environment = "production"
    Application = "iDempiere"
  }
}