CHAPTER 6 UNIQUE CONTRIBUTIONS OF AGILE MATERIALS TAKEN FROM SDLC 3.0 BY MARK KENNALEY INTRODUCTION • In software engineering no one model will fit all circumstances. • But the major models all contribute significantly to improve our processes. • As every enterprise is unique, so too much be the choice of major features of processes we have looked at. • Many of these characteristics may fit well to the current situation and constitute our process. 6.1 AGILE • Not long ago, there were huge needs for very rapid development of features and functionality; time to market could make/break a company. • Agile arose out of this environment. • While somewhat simmered, the features developed have transcended this period and still mark great strides in the improvement of the software engineering process. • We will consider the salient features. 6.1.1 TIME BOXING • Idea behind time-boxing is to establish a rhythm for development of value. • When rapid delivery of features was more important than quality, time-boxing arose! • This is a major feature of Agile and all iterative / incremental approaches. • Rather than deliver software of value be developmentdriven (scope boxing) and arrive at an uncertain cadence, forced times – with discipline – arose. • Typically they run from two-week iterations to a 30-day Spring in Scrum. • Emphasis: Speed of delivery is critical; cost of delay is great. TIME BOXING – CONTINUED • In time-boxing, initial objectives (scope) for the iteration are made solid prior to starting; roles and risks are firm. • Objectives are prioritized traditionally based on risk and need (provided core values to customer) • Within the iteration, there is continuous monitoring and controlling of the features to meet the date. • If it appears that some of the features will not be done (again, based on priorities), and this means the foresight for a working, tested increment of value and potentially shippable) then those features not accommodated are NOT used to extend the iteration, rather, they are postponed until a later iteration – to keep rhythm in tact. Features are scaled back. • An arrival of a ‘stable business value’ is reached, • demonstrated to stakeholders, • measure and assess the success (good, bad, and ugly) of the iteration, • revisit and re-rank the backlog (adding un-accommodated features if any), and get ready for the next iteration. TIME BOXING – CONTINUED • Important to note that ‘scaling back’ is somewhat ‘counter-intuitive and controversial’ (according to Kennaley) to many project managers. • They traditionally extend the work day or extend the delivery date (a slip) and ‘press on’ with additional hours (or people), and deliver based on ‘finished objectives.’ • If developing an Agile Project, the customer is IN THE LOOP, and his/her expectations will be in synch with time-boxing where delivered increments of real business value are provided. 6.1.2 – PRODUCT BACKLOG • A Product Backlog is simply a collection of product requests that are prioritized • The PB is a managed collection. • Many forms • Sticky notes on whiteboard (Agile) • Stack of index cards • Electronic versions • This effectively removes the need for a Change Control Board (CCB) PRODUCT BACKLOG – CONTINUED • Noteworthy: it is the Product Owner who is responsible for managing the backlog including submission, prioritization and obsolescence of items. • This is a fundamental paradigm shift on scope management and favors embracing the reality of Changing Scope rather than trying to control it. • Also implies less need for Analysts as team can now go directly to the source for demand requests. • Becoming less direct as there are many business stakeholders not able to fully articulate their needs, and • Analysts are needed through the realization that backlog items eventually require a more elaborate form such that acceptance testing can occur. THE PRODUCT BACKLOG • Product Backlog contains collected, prioritized, and allocated work items in time-boxed iterations. • In the PB, the high priority items (work items) must be well-defined as we enter an iteration. • Especially those with complexity or other meaningful constraints with high risk. • In the PB, low priority items may be vague, such as CRUD items. • Within an sprint, work items can be reprioritized / removed at any time, but must be well understood. THE SPRINT BACKLOG • Similar to the Product Backlog. • Used to manage work breakdown for an iteration/sprint. • What is needed for this iteration • This backlog is used internally by the team for coordination and management purposes and mid-iteration assessments. • Backlogs force us to prioritize work items. • This differs from much prescriptive work in the old CC backlogs. • Here, the belief is that there is much more fluidity in modern software engineering. • Product Backlog will likely never be empty until product is retired. • Thus, it reflects a Product Management versus a Project Management perspective. 6.1.3 BURN-DOWN AND VELOCITY • We term the rate of change (first derivative) of remaining scope per unit time as Velocity to establish the notion of how much ground or distance is being traveled. • Central idea is that we drive the Product Backlog to zero and use the measure of velocity to enable an extrapolation of the trend (the slope) to assess when we will likely complete the effort. • Note that we are addressing ‘delivered scope’ and not ‘work effort.’ • Work effort has a complicated relationship to scope. • Just because we are doing work does not mean we are progressing. • Work reflects the fuel being consumed. Could be at full throttle and still be going backward if headwind is too strong. BURN-DOWN AND VELOCITY - CONTINUED • Using basic calculus, the second derivative – or rate of change of velocity – will tell us if we are slowing down or speeding up. • Can take appropriate actions based on this info. • In planning an iteration’s work items, we can take the average of the velocity to address the scope of a time-boxed iteration as some stable indicator of what exactly we think can be accomplished within an iteration. 6.1.4 CO-LOCATION • A very controversial practice fraught with logistical difficulty. • This factor is often cited as the largest factor against scalability with Agile. • Often meets with resistance and cultural overtones. • Yet makes great sense – to enable the richest collaboration and communication possible. • Stats regarding communication effectiveness: • Words – 7% • Voice (pitch, speed, volume, tone) 38% • Visual (eye contact, body language) 55% CO-LOCATION – CONTINUED • Often some kind of compromise is reached despite the value of this core principle that provides fantastic communications. • In practice, it is oftentimes very difficult to establish, as many want to move into their ‘cave’ vice a ‘war room’ to be able to think. • Key Point: • If the great advantages of co-location are not possible, it does NOT mean the organization cannot do Agile. • It means that some degree of pragmatism must be inserted such that less than perfect strategies for achieving similar results need to take place. 6.1.5 SELF-ORGANIZING TEAMS • In stark contrast to command and control plandriven development • In Agile, there is no real Project Manager. • Closest role is that of a Scrum-Master – who is responsible for the process (whereas a Product Owner is responsible for the product. • The Scrum Master is more about facilitation dealing with sprint planning, coordination, issue or ‘block’ resolution when it comes to execution of a Sprint, and interfacing with management for reporting (if required from the process perspective). SELF-ORGANIZING TEAMS – CONTINUED • This practice hinges on Theory Y management, which states that employees ‘may be’ ambitious and self-motivated and exercise self-control. • This is opposite to Theory X Management, which many traditional managers practice) in which management ‘assumes’ employees are inherently lazy and will avoid work it they can, needing to be told what to do. • These contrasting philosophies surround the issue of workplace motivation and self-actualization and suggest that a ‘trusting management environment’ must exist for the phenomena of highly performing software delivery teams to occur. SELF-ORGANIZING TEAMS – CONTINUED • This software engineering practice was not invented by Agile, but has definitely been popularized through the Agile movement. • The military has done this for years. • Note: this does not mean that there are no managers • Management focus changes to be that of an escalation path for exceptional conditions where the team does not have the necessary visibility into the right course of action. SELF-ORGANIZING TEAMS – CONTINUED • Styles of governance tend to lead to discussions around the meaning of leadership, leaders, and management. • Here are a couple of definitions: • To manage: to handle or direct with a degree of skill: as to exercise executive, administrative, and supervisory direction. • To lead: to guide on a way, especially by going in advance; the process of social influence in which one person can enlist the aid and support of others in the accomplishment of a common task. • - Webster’s Dictionary. SELF-ORGANIZING TEAMS – CONTINUED • Often disagreements between the faithful of the PMBOK (Project Management Body of Knowledge) sometimes called the traditionalists, and Agile (sometimes called the progressives) in which claims of obsolescence of the Project Management role are made. • These roles / philosophies can be argued. • Bottom Line: existing Managers need to become Leaders, leveraging all the skills they have accrued over the years, shelving what does not reflect reality in modern software engineering, independent of any one method’s branding. • Make no mistake: management of large scale investments will still be the norm within business circles for some time to come with all the compliance and fiduciary duty legal issues. • But if common ground exists to help better articulate the real value proposition Managers can provide to IT Delivery teams, it is that of Leadership. 6.1.6 CONTINUOUS INTEGRATION (CI) AND RELEASE • This is a practice coming from XP community; frequently mentioned with Agile. • This practice is now a hallmark of the typical custom development project that is deemed Agile with most if not all Configuration Management (CM) vendors claiming CI capability. • Key concept: integrate & test early and often • But CI takes the concept to the extreme • with a per check-in build being triggered if configured in this default way, with at least daily check-in advised. • Issues typically arise when builds do not have enough time to finish for extremely large developments, or with product-line and multigenerational product management, so various strategies exist to support ‘almost’ continuous integration, with staged builds and variations in build scope. CONTINUOUS INTEGRATION (CI) AND RELEASE (CONTINUED) • So what happens in more robust parallel development strategies, integration builds and test triggers occur on a perdemand element, such as on a ‘feature, story, scenario, activity,’ so that “effective” integration occurs at a less frequent pace. • Is usually supplemented with the typical daily check-ins and local developer builds and test execution, along with potentially daily resynchronization enforcement with the mainline. CONTINUOUS INTEGRATION (CI) AND RELEASE (CONTINUED) • Along with the build trigger mechanism that ties CM infrastructure with build servers, unit test jobs are also triggered at a near continuous frequency, with the key goal to prevent defects from “breaking the build,” thereby ensuring project health. CONTINUOUS INTEGRATION (CI) AND RELEASE (CONTINUED) • Continuous / near continuous release is a little more contentious of a practice it turns out when dealing with most complex IT organizations. • With the Agile practice of Shippable Product, the intent is to have a complete increment of a product ready for a customer. • This means done-done within the Scrum community. • But, if you read the practice, it doesn’t necessarily mean that a release will occur, contrary to what some Agilist perspectives might say. • The key issue is the ability for the business customer to consume a release. • At whatever frequency realized • demand can be ”pulled” by the customer, • any deployment must be automated and scripted for repeatability and avoidance of human errors, • and integrate with Release Management orchestration. 6.1.7 PAIRING • Pairing is a practice popularized by XP. • It is where two developers sit at the same terminal and take turns developing software. • One is typing the code (driver) and the other is called the reviewer, who performs real-time code reviews. • Practice is linked to another agile practice called shared code ownership, where pairs rotate among teams such that shared, collective understanding of the entire code base is achieved. • Benefits claimed include • knowledge loss risk mitigation, • increased discipline and time management and • the concept of having two pairs of eyes rather than one to solve complex problems. PAIRING - CONTINUED • Your author asserts and wonders why this notion is restricted to developers. • Why not to other along different skill-sets. • Microsoft calls the concept of pairing along different lines “failure crews.” • A failure crew is a small, self-organizing team made up of a problem domain resource (analyst) and a solution-domain resource (developer). • This configuration is also referred to as a Lean work-cell. • Key skill-sets of the analyst are brought to bear to facilitate an understanding of demand through their knowledge of the problem space, and the soft skills necessary to elicit the tacit knowledge associated with the demand. • The Developer is responsible for leveraging their problem solving skills (design) in concert and concurrently with realizing a solution based on current technologies and technology constraints. • The key benefit of pairing along this way is that the team evolves through the Forming, Storming, Norming, Performing, and Adjourning team lifecycle. • They learn how to interact with each other in a non-silo’d way such that their slice of the software delivery value stream is efficient and effective. 6.1.8 TEST DRIVEN DEVELOPMENT (TDD) • Another example emerging from the Agile community is known as Test Driven Development. • Precedent for this practice comes from XP and Kent Beck. • TDD is realized through the idea of having developers design unit tests (as opposed to functional or acceptance tests) before writing code. • These tests are usually developed using unit test harness tools such as Junit, FitNesse, etc. • These unit tests are integrated with Continuous Integration infrastructure such that they are executed every time code is checked in to a common source code repository. TEST DRIVEN DEVELOPMENT (TDD) CONTINUED • Similar pre-cursors to this practice have existed in relation to certain object-oriented languages. • One such language, Eiffel, invented by Bertrand Meyer. • A manifestation of guiding developers to think about failure conditions before writing algorithms (thereby abstracting interraces, either type-based toward the specific, or instance-based towards the generalized specification).