With pervasive uptake among companies, digital transformation is the latest, hottest trend in the tech space — despite being one of the least defined in recent history.

In practice, digital transformation is often used more as a description of what companies are trying to do, rather than as a concrete methodology for how they are doing it. Without the how — you have change for change sake — versus what the Japanese call kaizen, Good Change. Like a doctor who can not hurt a patient, digital transformation must yield improvement.

Given this, it’s worth trying to fit digital transformation into the larger historical trends of technology. In a recent podcast with Howard Morgan, noted venture capitalist, philanthropist, researcher, and author, I did just that.

We reviewed the history of many technological transformation methodologies to see what we can learn about how digital transformation will fit into this larger picture. Hopefully you will find insights and create an approach that truly transforms your organization.

Defining  Digital Transformation

How we define digital transformation is highly dependent upon the business that is trying to change. In my studies, I’ve found that how companies think about it depends on the problems they need to address — and those range from a data problem, where they have to figure out how to use more data in more ways than ever before, to an integration problem.

This includes bringing together multiple silos to create a relationship- or product-centric view, and then expanding that internal integration into an external integration of ecosystems that involves third parties, such as business partners or suppliers.

Digital transformation also can be  a cultural and organizational change, where the goal is to change the way the business uses technology to power operations.

In other cases, companies think of digital transformation as agility. Being able to move fast in smaller increments allows organizations to gather data through techniques such as A/B testing, and that, in turn, allows them to make better, faster business decisions.

Seeking to Reshape Business Interactions

But it could also be seen simply as a business model to keep up with technology. And, if we’re being honest, we have to admit that often, what most people really have in mind is digital transformation as imitation. They want to duplicate the successes that they’ve seen in Amazon, Netflix, Google, and Facebook and create new agile and evolutionary ways of doing business with technology and new methods — and make a lot more money.

Regardless of how it’s viewed, digital transformation always seeks to reshape the way businesses interact with and use technology. Morgan believes the data aspect of digital transformation is the key, and companies must understand and apply their data better than ever before to succeed.

“Digital transformation is the understanding that if you can get all the information about all the things that are happening to your business into a database form, into digital form, you can then manipulate that in ways that allow you to manipulate the real world more efficiently, and spend less money.

“You can put things together that you couldn’t put together before. It could be in manufacturing processes, where we’ve seen, over the last 30 years, supply chains collapsing, because you have the information about where inventories are at your suppliers, so then you can get to the kaizen, you can get the just-in-time, integrating the outside world,” he said.

In other words, the data leads to a higher resolution understanding of your business as a whole. The model leads to insights and opportunities for automation. These new superpowers lead to better business models, products, and services that evolve, based on data from experience, faster than ever before.

Parts of this larger pattern have emerged in past trends. In my view, mastery of what digital transformation could mean to an organization can be improved by looking at history and examining how other technological trends have changed the enterprise. In my conversation with Morgan, we covered a number of such changes and the lessons they offer for us today.

Here’s the arc of our conversation:

  • Business process re-engineering is a trend that took hold in the 1990s and was focused on refactoring processes to make them more efficient, a common theme of many digital transformation efforts.
  • Total quality management, lean manufacturing, and Six Sigma held sway from the 1980s to the 2000s and increased use of instrumentation and metrics in the service of removing variation and defects from processes. Originally applied to manufacturing and process control, these methods became widespread, with GE turning them into a management philosophy. Digital transformation efforts often dramatically expand the use of metrics in the same way.
  • Agile, scrum, and lean startup took hold in the 1990s and 2000s and are in wide use today. These methods focused on making incremental progress, getting evidence about what is working, and gradually steering toward success — approaches widely adopted in digital transformation programs.
  • Finally, we discussed the use of objectives and key results (OKRs) and how Amazon has focused on defining its business in terms of services. Both methods are used to express a grand plan across a business and keep the focus on making coordinated progress in a distributed organization. Digital transformation programs need ways to achieve focused collaboration at scale.

Business Process Re-engineering

According to Morgan, business process re-engineering was all about helping optimize the enterprise. It arose in an era where the integration of suppliers and companies was just beginning.

This led to new processes around ordering, supply chain management, and ERP. In addition, sales teams used tools, such as early versions of Salesforce, to manage and track how they interacted with customers — and then re-engineer those processes to focus the right salespeople on the right customers with the right metrics.

“Business process re-engineering took us a little bit of the way, but it was two or three links in the chain from a ‘raw material,’ whether it’s data information raw material or physical raw materials, to the product and service delivered to a customer. Now, with what people have called digital transformation, they want to go the whole way. They don’t want to just re-engineer a piece of the process; they want to re-engineer the entire process end to end,” said Morgan.

By thinking and redesigning processes, along with the IT systems that were supporting them, enterprises are able to improve agility and flexibility and make change easier.

However, business process re-engineering didn’t work in many cases where the underlying IT infrastructure and applications needed to be dramatically transformed. The technology of the era could not accommodate such changes, and re-engineering efforts suffered.

Total Quality Management, Six Sigma and Lean Manufacturing

The next development I discussed with Morgan was total quality management from pioneering statistician W. Edwards Deming. Many ideas fell underneath the umbrella of total quality management, but the main one was that the highest quality product was also the lowest cost product.

If that seems like an oxymoron, it’s not. Deming’s idea was focused on quality at every level and trying to make sure that no stone went unturned in an effort to save money by improving quality and spending less time on correcting mistakes.

Inspired by Deming, Toyota developed its Toyota Production System. Described at the time by American academics as “lean manufacturing,” the system has many powerful practices including pull-based production, just-in-time manufacturing, and of course, kaizen.

Manufacturing first adopted lean manufacturing, but it then became generalized in many industries and has fed directly into the more recent ideas of lean startups and the entrepreneurial operating system.

A more general method was Six Sigma, which promoted the idea that you could use statistical methods to remove variations and reduce mistakes that you didn’t have to go back and fix later.

Both lean and Six Sigma rely heavily on statistical quality management. Jack Welch famously implemented Six Sigma at GE, which helped propagate these practices beyond the manufacturing realm.

“It really was true that if you had high quality, then you could have lower costs because you didn’t have redos, scrap, or wastage, and that quality came through huge amounts of inspection of everything moving into the process, huge amounts of statistical quality control — whether it was bending a piece of sheet metal or soldering components onto a printed circuit board.” Morgan said.

In other words, monitoring components during the manufacturing process increased yields and, effectively, decreased costs. “That was a revolution on the manufacturing side,” Morgan said.

Agile, Scrum, and The Lean Startup

In the world of software development, Agile methods and Scrum created a revolution by replacing what has been called the waterfall method with an incremental process that was skeptical about the quality of requirements. Instead of spending lots of time creating detailed requirements, in agile and scrum projects, a basic version of the product is created and used and then gradually made better in small steps.

Eric Ries’ Lean Startup took ideas from agile, scrum, and lean and built on them with a central idea of, as Morgan said, “move fast, build a minimum viable product, incrementally improve, and do that over and over again in a very rapid cycle.”

“These approaches started to be adopted by very large companies in innovation, allowing their people to start building changed systems, new systems, particularly when they had information that they could use to interact with customers. For example, one of the big companies I’ve worked with over the years is John Deere, which started having their folks use agile software development in their finance insurance agencies and John Deere finance companies about 10 or more years ago.

“It completely transformed how quickly they could respond to changes in the needs of farmers and dealers as the company put out new financial products. They used agile development and scrum methodologies and that meant that they could run through and test and put into production four or five times a year, versus once a year or two-year cycles for IT that were the norm before that. It has had a major impact,” Morgan said.

However, what I believe is lost in this evolution is that companies cannot iterate to a perfect design or perfect product. There is still a design sensibility that needs to guide the company. Without that design vision, companies don’t know the limits of what they can pull together.

Objectives and Key Results (OKRs)

One of the final developments Morgan and I talked about were OKRs. “You set three goals with measurable outcomes for the next quarter, goals that, if achieved, will significantly advance the organization’s objectives. An OKR might be to launch a particular product or to get a certain number of customer wins if you’re on the sales side,” Morgan said.

“And then you look at what key results are needed along the way so that you can monitor them during the quarter and review them with your manager at regular intervals, monthly or even weekly. And then everybody in the organization looks at them all at the end of the period and says, ‘Which ones did we achieve, which ones didn’t we achieve, and what do we have to change?’”

“This keeps people focused on a few key objectives, and when they’re doing something, they can always look up and say, ‘Is what I’m doing today helping me meet my OKRs for this quarter? And if it isn’t, why am I doing it?’”

Thus, OKRs solve the design vision problem for the organization that I mentioned earlier — they are a straightforward way of expressing what a company should do. And this has particular resonance for digital transformation. In companies where I’ve seen digital transformation efforts fail, it’s because those businesses are weakest in understanding their larger vision. The OKR method could be used profitably to help correct this.

Amazon’s Service-Based Approach

Finally, Morgan and I discussed how Amazon has applied the idea of APIs to the business side of management. “What Amazon did was make not just the coding in that organization impact the way they approach APIs, but everything the organization did, whether it was human resources, acquiring hardware, or warehousing goods so that they could then externalize warehouses.

“That all came about from that change, which was not forcing everybody to deal with every other part of the organization, because that would have led to chaos, but to create interfaces, APIs, that anyone could use internally and later on, with AWS and Amazon storage, externally. And that meant that they could really expand the company, which obviously, Amazon’s done in spectacular ways,” Morgan said.

Ultimately, what Amazon’s approach shows is that if the business can encapsulate a particular piece of the organization and make interfaces available to the rest of the organization, the faster a total transformation can occur. The result is that everyone from internal staff to customers can interact with the business in a more efficient way that cuts capital costs, wasted time, and personnel expenses. Digital transformation efforts often seek to allow a distributed organization to move as fast as possible at the edge, while maintaining order across the landscape. Amazon’s model provides one example of how to achieve this difficult goal.

Lessons for Digital Transformation

This tour of methods shows that the business world has always had a strong appetite for new ways to change, transform, and improve. The examination of these methods, and how each one relates to digital transformation, shows that while each of these historical trends has important lessons, no digital transformation program can apply to all of them at once. Each business must understand what digital transformation means to them and select ways of transformation that fit their goals.

Dan Woods is a technology analyst, writer, IT consultant, and speaker. He is the founder of EarlyAdopter.com and Evolved Media. Dan’s mission is to help people find the technology they need to succeed.