Collaborate 13 - Publish Services: A Mile-High View
By Brian Diehl on Apr 09, 2013
This is my first post and it is coming, appropriately, from Collaborate 13 in Denver. Conferences always spawn new ideas for me and the idea for this blog came after OpenWorld 2012. I'm here to present a “Quick Start” guide to Reporting and Analytics. No easy task. These are truly enterprise products with many moving pieces. So I've decided to take a different approach. My presentation at Collaborate 13 is just the start of a longer term process of which this blog is a major part. This post will kick things off with the "mile high view" of the pieces making up Primavera Reporting and Analytics (PR&A) starting with Publish Project.
PR&A truly begins with Publish Project Services which we call PX. PX was a result of a customer interaction at OpenWorld a few years ago. He questioned why Primavera did not store all data in the transactional database like “everyone else”. A fair question. At the same time we were struggling with the same challenges when it came to the relatively new Enterprise Reporting Database and Star schema. In both cases, the lack of important fields in the transactional database cause a lot of processing during the ETL (Extract Transform Load). Hence, PX services were born.
In simple terms, PX is a way to extend our current schema and fill in the missing data in the most performant way. Instead of spending years re-architecting P6 to store everything as physical fields, costing more system resources on each update, the PX services perform the same function a bit more efficiently.
PX has three core components. The first is extended tables we call X-tables. X-tables are primarily used to extend some base table in P6. The TASKX table adds additional columns to the TASK table. These X-tables have a one-to-one relationship to the base table.
A group of services, on the P6 application server, are the second element to PX. The P6 job service infrastructure executes several custom jobs to incrementally recalculate the data in the X-tables. Central to everything is the Publish Project Service. This service is really not a single service, but multiple jobs for each project. It performs the bulk of the processing effort and is the most user-facing element in the entire process.
And that brings us to the final, central component to PX services: the Arbiter. The Arbiter is a way to simply and elegantly (I hope) queue projects to be recalculated. This is done based on 1) demand by a user to recalculate now, 2) enough rows have changed in the project since the last recalculation and finally 3) enough time has passed and there was at least one update.
That's was a very high-level overview of Publish Project. Look to this space in the future for details on this and more.