Case study · Commercial

Real-time, change-tracked data pipeline for a private equity platform

Bithrive engineers designed a real-time pipeline around FIS Private Equity Suite (Investran), built automated workflows and added AI invoice processing.

The challenge

Nightly batch jobs slowed reports, reporting reads competed with writes at quarter close (the most demanding time of year), and manual steps in day-to-day operations took time away from the business.

What was delivered

  • Custom, automated workflows that improved the firm's business operations.
  • A change-tracked, real-time pipeline from the FIS Private Equity Suite (Investran) transactional database into a separately designed, read-only star schema.
  • Enterprise service bus integrations with upstream and downstream systems, with queueing, throttling, monitoring, and synchronization.
  • The enterprise data warehouse, OLAP cubes, and existing reports kept supplied while the platform modernized.
  • Business APIs for a unified user experience alongside the legacy front ends.
  • AI invoice processing for accounts payable (2024 to 2026): OCR and AI models on Amazon Bedrock extract invoice data, and staff review the results.

Upstream and downstream systems

Upstream: feeds the platform

  • Appian
  • Salesforce
  • PeopleSoft
  • Workday

Downstream: fed by the platform

  • Appian applications
  • Enterprise service bus subscribers
  • Enterprise data warehouse
  • OLAP (MDX) cubes
  • SSRS, Crystal Reports, and Excel reporting

How it works

Not SQL Server replication. SQL Server’s built-in replication copies tables as they are. This platform does something different. It uses SQL Server Change Tracking to detect each change in the FIS Private Equity Suite (Investran) transactional database, and a custom-built engine transforms that change and applies it to a separately designed, read-optimized star schema, in under 10 milliseconds. The reporting database is shaped for the questions analysts and downstream systems ask, not a copy of the source tables.

Change-tracked pipeline into a custom star schema Changes in the Investran transactional database are detected by SQL Server Change Tracking, transformed and applied by a custom-built engine with queueing and throttling, and written to a read-optimized star schema that serves reports, OLAP cubes and enterprise service bus subscribers. Investran transactional DB Change Tracking detects each change Custom engine transform and apply queueing, throttling Star schema read-optimized replica Reports, OLAP cubes, ESB subscribers under 10 ms end to end Change-tracked pipeline into a custom star schema Changes in the Investran transactional database are detected by SQL Server Change Tracking, transformed and applied by a custom-built engine with queueing and throttling, and written to a read-optimized star schema that serves reports, OLAP cubes and enterprise service bus subscribers. Investran transactional DB Change Tracking detects each change Custom engine transform and apply queueing, throttling Star schema read-optimized replica Reports, OLAP cubes, ESB subscribers under 10 ms end to end
Each change is reshaped as it moves, so the replica is built for reporting rather than copied from the source tables.

Approach

Read/write separation removed the contention at its source: the transactional database takes writes, and the replica serves everything that reads. Integrations run through the enterprise service bus with queueing and throttling, so a slow or unavailable system does not hold up the rest. New experiences were introduced beside the legacy screens, so users moved over gradually with no big-bang cutover.

Outcomes

  • Data reaches the replica in under 10 milliseconds, including at quarter close.
  • Read/write contention eliminated and nightly batch jobs retired.
  • Users moved to the new experience without a big-bang cutover.
  • AI invoice processing with human review: 40% less manual work and 70% less time per invoice.

Let's discuss your next project.

Tell us what you are trying to change. We will reply with practical next steps, whether or not we turn out to be the right fit.

Discuss Your Project