Upstream profitability is challenged if oil markets remain oversupplied and prices modest. The solution is a sustained focus on deep structural change. There's much Upstream operators can learn from Downstream, which has emerged thriving from the doldrums of a decade ago.

Downstream has been the poor cousin of the oil and gas value chain since the Great Financial Crisis. Some regional pockets have fared better than others, high oil demand growth economies of Asia, or parts of the USA where a host of factors conspired to create cheap feedstock. But for much of the period since 2008 there has been too much refining capacity, and refiners struggled to eke out a living.

How times have changed. The Majors' latest results confirm another generally solid year from Downstream, the third in a row of significant outperformance over Upstream. In the early 2010s, Downstream contributed around 18% of the Majors' 'clean' earnings (excluding one offs); Upstream the very dominant balance. The relative shares of group earnings have completely reversed in the last three years, during which Downstream profits have all but doubled whilst Upstream has more than halved.

Downstream should have another decent year in 2017 – Alan Gelder, our Senior Refining and Chemicals Analyst, expects global margins to be down on 2016 but above average.

These swings and roundabouts are what integration is all about. Downstream is a hedge through the cycle, capturing a bigger margin, usually ephemeral, as oil prices fall. The Majors and other integrated companies have always valued some balance between Upstream and Downstream, though the weighting between the two shifts over time. There has been a significant drive to reduce capital employed in Downstream and focus on Upstream this decade. Some integrated companies, like ConocoPhillips and Marathon, have taken reweighting to an extreme, exiting Downstream entirely to become pure Upstream players.

But the improved profitability of Downstream is down to more than a mere arbitrage between feedstock and refined product and petrochemical prices. Deep structural change wrought over several years is now enabling refiners to reap the benefits of a cycle swinging their way.

So what exactly have the companies done with Downstream to get them into this 'sweet spot'?

First, reduce capacity and portfolio high grading. Concentrating the portfolio around advantaged assets takes time but has been a critical step towards boosting ROCE. The majors' refining capacity has been cut by 20% since 2010, with old, competitively weak assets sold off or closed down. The process continues, with Shell disposing US$5 bn of Downstream assets in the last year alone, and looking to unwind its US Motiva JV with Saudi Aramco. US Majors have tended to retreat to the US, Europeans into Europe.

Second, tight capital discipline - not merely as a knee jerk reaction to conserve cash but as an essential component of a returns-driven strategy for the longer term. Cutting capital investment has its place when times are tough, but can only be a short-term solution for any business, as upstream will find in time. The trick is to use available capital to make the most of core assets; to position in advantaged growth markets; and develop technology that differentiates or gives a cost advantage.

Third, manage for margin. This has meant aggressively running the business at the lowest cost, safely - to maximise margin as a price taker. That focus never ceases for the best Downstream operators. It's about a deep understanding of the hydrocarbon and equipment supply chain; allocation of human capital and right-sizing; analysing individual refinery/petrochemical asset performance and profitability; managing operations and processes more efficiently, including digitalisation and technology. Big data, predictive analytics, automation and visualisation are becoming increasingly central to Downstream performance, reducing costs, improving safety and lifting returns.

The trigger for Downstream operators was recognising that there was a chronic problem with through cycle profitability and acting – and that didn't happen overnight.

The optimists in Upstream anticipate a tightening of oil markets and higher oil prices that will restore profitability. The more sober view is continued oversupply and modest oil prices that would put a premium on high-quality, low-cost portfolios that are 'managed for margin'.

Upstream companies could do worse than follow the Downstream template. The advent of CEOs in some Majors steeped in the Downstream experiences of recent years increases the chances of that happening.