Welcome!

Agile Computing Authors: Liz McMillan, Elizabeth White, Ed Featherston, William Schmarzo, Pat Romanski

Related Topics: Microservices Expo, Java IoT, Industrial IoT, Microsoft Cloud, IoT User Interface, Agile Computing

Microservices Expo: Article

The Odd Couple: Marrying Agile and Waterfall

Interfacing between Linear Waterfall and Agile Approaches

This article depicts the best practice approach for integrating Agile approaches and specifically Scrum development with traditional overarching linear approaches, specifically waterfall methodology. The agile PMO, properly defined, can be positioned to secure Agile-Scrum benefits while maintaining the necessary overarching control.

The challenge
Over the last two decades, various Agile approaches have been introduced and practiced. Of these, in last 5 to 7 years, Scrum has gained the most popularity resulting from a combination of simplicity, ease of use, and effective public relations. Scrum success in software development organizations has been a powerful driver for roll outs across products, industries and businesses. As described, this was exacerbated by a focused marketing effort from Scrum evangelists. Unfortunately, most of these organizations were not structured in a way that supports the Agile-Scrum approach and methods. Even more so, scrum in its raw and pure form is not suitable for the majority of organizations.

The first wave of failed Agile-Scrum implementations brought about an even stricter admonition, based on an unwavering belief from Scrum zealots. The main assertion has been that failed implementations are a result of partial embracing of the true scrum spirit and the full benefits of the approach can only be attained if the entire organization is reengineered. This fanatical attitude left many project teams across organizations big and small, struggling with their already idealistic implementations. Some have been figuring out on their own, how to combine the contemporary and traditional worlds. Other teams have completely abandoned the Agile-Scrum concepts reverting back to the traditional linear waterfall approach and method. Yet other teams, ridden with guilt, manage Scrum by name only, and hiss vehemently at any project management proponent who is unfortunate enough to advise on re-embracing Agile in a more cognizant approach.

The concepts which are presented and embodied in Agile-Scrum are too good to be ignored; however implementing it outside pure software development requires adaptation.

Complex scenarios for Agile
The main hurdle in achieving the benefits of Agile- Scrum outside software development is integrating it with existing control mechanisms. These control mechanisms are stipulated due to various organizational prerequisites and are normally actualized by using the Linear Waterfall approach and methodology. Four of these typical organizational prerequisites are depicted below:

  • Big global corporates: in these hierarchical matrix organizations, top down portfolio control is the rule of the day. The free spirited agile approach has a tough time adjusting to the rigorous controls. Specifically the inherent Agile plan-free concepts, make Agile-Scrum difficult for the organization to swallow.
  • Highly regulated industries: some industries are required by compliance and governance bodies to have a strict binding control mechanism. These can be for example medical equipment, aircraft, and pharmaceuticals research and product development business units. While individual teams might operate Agile-Scrum, the development process must follow a rigid Linear Waterfall approach method for traceability and governance.
  • Complex predefined products: some integrated products which include both hardware, software, imbedded and more are developed as a contract with an end customer under a predefined set of requirements. In these cases the degree of requirement flexibility is small, though larger than what is anticipated initially. The concept of a fully flexible backlog of Agile-Scrum suffers considerably in these cases.
  • Generic IT departments: much of the daily and weekly activities in maintenance driven IT departments is ad hoc. Changes to the daily schedules are numerous and immediate. Constant interferences in the teams work are the norm. The concept of time boxing and no interference is difficult to maintain in these situations.

Naturally - many times the four discrete categories detailed above, actually mix; so it is common to find a complex product in a global big corporate which is required to comply with firm regulation.

Based on practical experience, the recommended method to tackle these scenarios and others is by empowering the Agile PMO to act as an enabler, driver and translator between the emerging Agile-Scrum teams and the Linear Waterfall elements.

Refer to the table below for specific guidelines

The Agile PMO - leading the hybrid organization - guidelines

Scenario

Challenge

Possible solution

Comments

Insights

Big global corporates

Strict controls manifested in Linear Waterfall

The Agile PMO is the buffer between Agile-Scrum teams and the Linear view

Burn down charts are translated to phases for control;

Requirement traceability done by PMO architect;

Agile PMO maintains the dictionary between sprint planning, execution and the phase gating mechanism

Product owners can be part of the Agile PMO;

Project initiating and closing managed by the PMO

Scenario

Challenge

Possible solution

Comments

Insights

Highly regulated industries

Strict compliance and paper trail requirement including product risk analysis

The Agile PMO is also resourced by administrative staff to ensure compliance with regulations

Product risk is managed on a lifecycle view with members of the Scrum-Agile team;

Backlog populated by Non-functional yet critical requirement and owned by the Agile PMO

Agile PMO staff maintains traceability of these requirements.

Necessary documentation is part of the backlog

The added administrative effort handled by the PMO is compensated by the increased velocity of the Agile teams.

Administrative PMO staff can also be non-functional product owners to ensure compliance aspects

Scenario

Challenge

Possible solution

Comments

Insights

Complex predefined products

Limited flexibility in product scope tends to deteriorate  Agile implementations to Agile by name only; Also, hardware elements of product can't be performed in an Agile approach

The Agile PMO owns the backlog interfacing with the various components of product development - managing a hybrid Agile-Linear project

This is probably the most difficult and tricky scenario to tackle;

It requires technical as well as leadership propensity and know-how.

Experience shows that by investigating creatively - Agile concepts can be implemented in rigid hardware development environments

Also - rigid product requirements still allow usually 20% flexibility

the most value added can be reaped in this scenario by developing a customized mixed approach;

Agile stage deliveries can be used to increase flexibility.

Concepts of incremental deliveries may sometimes not be achievable in all product aspects

Scenario

Challenge

Possible solution

Comments

Insights

Generic IT departments

Constant changes to team's work, inability to see the big picture due to ad-hoc work interfering;

missing a true product owner

The Agile PMO substitutes the product owner role in acting as a buffer to oncoming requests also protecting effort to reasonable levels

Many disheartened IT departments have become bitter when trying to use Agile to their development and ongoing work; the result has been fatigue laden team, viewing Agile as a vicious manipulation to increase output without genuine management support; more than a single project management approach can be practiced

Noticeably, Kanban works better for these environments;

Time boxing still makes sense, however a certain predefined buffer for ad-hoc work should be built into each sprint; Sprint durations should be flexible

 

 

Important best practices to remember that go hand in hand with the Agile PMO:

  • Implementing Agile-Scrum as a restricting admonition is exploiting the adaptive nature of Agile
  • There is no one right way - no one size fits them all;
  • There is no silver bullet - you can create what works for you;
  • Being agile and adaptive also allows being flexible with how one uses the methods, process and Methodology;
  • Time boxing is great as long as you are flexible to change the durations of the time box if necessary;
  • Sometimes the client isn't directly available, in these cases marketing and product management are a proper alternate;
  • Arbitrary rules don't complete projects, people do! Empower your team and yourself to choose the appropriate mix of approaches that enable product delivery.

Summary
In my book about the Agile PMO I describe how PMOs can succeed only if they create and enhance value through smart portfolio selection (more about that in a future white paper). With the emerging of Agile approaches and specifically Scrum methods new opportunities have become apparent. Integrating them into an existing control structure - typically presented by a waterfall lifecycle - can be frustrating. We have defined a new key player - the Agile PMO which can be positioned to create a transformation / translation layer between the approached and methods, contributing to higher success levels of these integrations.

 

The Agile PMO Michael Nir

More Stories By Michael Nir

Michael Nir - President of Sapir Consulting - (M.Sc. Engineering) has been providing operational, organizational and management consulting and training for over 15 years. He is passionate about Gestalt theory and practice, which complements his engineering background and contributes to his understanding of individual and team dynamics in business. Michael authored 8 Bestsellers in the fields of Influencing, Agile, Teams, Leadership and others. Michael's experience includes significant expertise in the telecoms, hi-tech, software development, R&D environments and petrochemical & infrastructure industries. He develops creative and innovative solutions in project and product management, process improvement, leadership, and team building programs. Michael's professional background is analytical and technical; however, he has a keen interest in human interactions and behaviors. He holds two engineering degrees from the prestigious Technion Institute of Technology: a Bachelor of civil engineering and Masters of Industrial engineering. He has balanced his technical side with the extensive study and practice of Gestalt Therapy and "Instrumental Enrichment," a philosophy of mediated learning. In his consulting and training engagements, Michael combines both the analytical and technical world with his focus on people, delivering unique and meaningful solutions, and addressing whole systems.

@ThingsExpo Stories
So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, provided tips on how to be successful in large scale machine learning...
Early adopters of IoT viewed it mainly as a different term for machine-to-machine connectivity or M2M. This is understandable since a prerequisite for any IoT solution is the ability to collect and aggregate device data, which is most often presented in a dashboard. The problem is that viewing data in a dashboard requires a human to interpret the results and take manual action, which doesn’t scale to the needs of IoT.
What does it look like when you have access to cloud infrastructure and platform under the same roof? Let’s talk about the different layers of Technology as a Service: who cares, what runs where, and how does it all fit together. In his session at 18th Cloud Expo, Phil Jackson, Lead Technology Evangelist at SoftLayer, an IBM company, spoke about the picture being painted by IBM Cloud and how the tools being crafted can help fill the gaps in your IT infrastructure.
"C2M is our digital transformation and IoT platform. We've had C2M on the market for almost three years now and it has a comprehensive set of functionalities that it brings to the market," explained Mahesh Ramu, Vice President, IoT Strategy and Operations at Plasma, in this SYS-CON.tv interview at @ThingsExpo, held June 7-9, 2016, at the Javits Center in New York City, NY.
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
Traditional IT, great for stable systems of record, is struggling to cope with newer, agile systems of engagement requirements coming straight from the business. In his session at 18th Cloud Expo, William Morrish, General Manager of Product Sales at Interoute, outlined ways of exploiting new architectures to enable both systems and building them to support your existing platforms, with an eye for the future. Technologies such as Docker and the hyper-convergence of computing, networking and sto...
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
It’s 2016: buildings are smart, connected and the IoT is fundamentally altering how control and operating systems work and speak to each other. Platforms across the enterprise are networked via inexpensive sensors to collect massive amounts of data for analytics, information management, and insights that can be used to continuously improve operations. In his session at @ThingsExpo, Brian Chemel, Co-Founder and CTO of Digital Lumens, will explore: The benefits sensor-networked systems bring to ...
SYS-CON Events announced today the Enterprise IoT Bootcamp, being held November 1-2, 2016, in conjunction with 19th Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA. Combined with real-world scenarios and use cases, the Enterprise IoT Bootcamp is not just based on presentations but with hands-on demos and detailed walkthroughs. We will introduce you to a variety of real world use cases prototyped using Arduino, Raspberry Pi, BeagleBone, Spark, and Intel Edison. Y...
Large scale deployments present unique planning challenges, system commissioning hurdles between IT and OT and demand careful system hand-off orchestration. In his session at @ThingsExpo, Jeff Smith, Senior Director and a founding member of Incenergy, will discuss some of the key tactics to ensure delivery success based on his experience of the last two years deploying Industrial IoT systems across four continents.
Much of IT terminology is often misused and misapplied. Modernization and transformation are two such terms. They are often used interchangeably even though they mean different things and have very different connotations. Indeed, it is somewhat safe to assume that in IT any transformative effort is likely to also have a modernizing effect, and thus, we can see these as levels of improvement efforts. However, many businesses are being led to believe if they don’t transform now they risk becoming ...
Identity is in everything and customers are looking to their providers to ensure the security of their identities, transactions and data. With the increased reliance on cloud-based services, service providers must build security and trust into their offerings, adding value to customers and improving the user experience. Making identity, security and privacy easy for customers provides a unique advantage over the competition.
SYS-CON Events announced today that Venafi, the Immune System for the Internet™ and the leading provider of Next Generation Trust Protection, will exhibit at @DevOpsSummit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Venafi is the Immune System for the Internet™ that protects the foundation of all cybersecurity – cryptographic keys and digital certificates – so they can’t be misused by bad guys in attacks...
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
"Tintri was started in 2008 with the express purpose of building a storage appliance that is ideal for virtualized environments. We support a lot of different hypervisor platforms from VMware to OpenStack to Hyper-V," explained Dan Florea, Director of Product Management at Tintri, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
"There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
For basic one-to-one voice or video calling solutions, WebRTC has proven to be a very powerful technology. Although WebRTC’s core functionality is to provide secure, real-time p2p media streaming, leveraging native platform features and server-side components brings up new communication capabilities for web and native mobile applications, allowing for advanced multi-user use cases such as video broadcasting, conferencing, and media recording.
IoT generates lots of temporal data. But how do you unlock its value? You need to discover patterns that are repeatable in vast quantities of data, understand their meaning, and implement scalable monitoring across multiple data streams in order to monetize the discoveries and insights. Motif discovery and deep learning platforms are emerging to visualize sensor data, to search for patterns and to build application that can monitor real time streams efficiently. In his session at @ThingsExpo, ...