Americas Aerospace Quality Group AAQG Project Management AAQG Project #61 (IAQG REQ-009) IAQG 9101:2009 Revision - Quality Management System Assessment Reported by: Tony Marino Date of Report: 2009-March-11 What happened and where is 9101? • Released in October 2008 for ballot – AAQG writing team members did not agree it was ready. – Consulted with AAQG leadership to release or move for sector rejection – decision for sector rejection. • Primary Issues – Writing was a balanced writing addressing both Conformity and Effectiveness. – Writing structure – too confusing. – Contained guidance material. – Differences of opinion on what could be changed during and after ballot. • Document revisions have been made and process of finalization for ballot. IAQG REQ-009 Working Group Members Hans Luijt – Team Lead Stuart Anthony Eddy Benattar Alain Bonnel Conny Flink Michel Jacquaniello Marie-Laurence Petit Brian Burroughs* Michel Clément* Larry Beck Tom Gallagher Tony Marino Robert Keys* Paul Dionne* Masahiro Kawamoto ASD-EASE - Netherlands Rolls-Royce UK IAI - Israel SAFRAN - France Volvo Aero - Sweden Zodiac - France Airbus - France LRQA – UK Bureau Veritas - France Lockheed Martin – US (TX) Hamilton Sundstrand (UTC)-US The Boeing Company – US (WA) DNV – US (TX) Intertek Mitsubishi Heavy Industries – Japan 9101: Revision Objectives • • Ensure alignment with IAQG strategy (on-time, on-quality performance, Strengthen of Other Party certification) Support the process approach for quality management systems as described in the 9100-series standards • Incorporate ISO 17021 changes and ICOP resolutions • Define requirements for all activities of audit program • Define adequate audit report that provides useful information to stakeholders on Organization Processes Performance And Effectiveness ( PEAR ) and conformity (Objective Evidence Record) to 9100 series . • Define audit records (objective evidence,…) to support ICOP Scheme Americas Aerospace Quality Group - Project Plate AAQG P61 - IAQG 9101:2008 Revision - Quality Management System Assessment Date of project approval: AAQG Project Leader: Champion: Tony Marino Jack Flecher Project Plate Revision: 2006-Oct-05 Problem Statement and Scope: Deliverables: IAQG Gate 0 Approved October 5, 2006 - Seville Spain (REQ-009) 1. Revised 9101 1. Current 9101 does not focus 9100 assessments on process based management 2. Develop position and decision to obsolete 9111 and 9121 2. Too much emphasis on conformity of assessment – not auditing what is important i.e. performance, continuous improvement, customer satisfaction 3. Aerospace Experienced Auditor training and certification/ recertification recommendations 3. Linkage to certification: certificate too much of a ‘drivers license, instead of a tool to prevent ‘accidents’. 4. Scoring is complex, time consuming, subjective and purpose of is unclear. 5. Strong relationship of assessment documents (9101, 9111 and 9121). Root Causes: Success Criteria: 1. Upon release of ISO9001:2000, assessment tool and methods for 9100 was not updated to assess the process approach. 1. Publication of 9101 2. New 9100 release in April 2009 will require update of 9101 3. Greater fidelity of information in audit reports Solution: Metrics: 1. Update 9101 to be more than an assessment (checklist) document. 1. Project Milestone 2. Publish assessment methods and tools that will provide more meaningful information to the supplier, client and stakeholders 3. Consolidate 9101, 9111 and 9121 into one document 2. Auditor efficiency increase )6/20/08 Americas Aerospace Quality Group - Project 4-Square Date of last report: 2008-Sept, 11 P61 - IAQG 9101:2008 Revision - Quality Management System Assessment Project Leader: Date of this report: AAQG Champion: Status: Y 2009-March, 11 Project description / benefits: Tony Marino Kristy Heffernan Recent accomplishments (since last report): Project Description • • Restructure 9101 with the next revision of 9100 to combine the 9101/9111/9121 standards to reflect a process based auditing approach of 9100, 9110 and 9120 standards. • AAQG SWOT and sector rejection of ballot document Nov 23, 2008. • Training specification and competency requirements development Nov 17-20, 2008 – Paris Auditor training and competency requirements development. • Team F2F to address issues and actions. Huntington Beach CA. Jan. 2009. • Proposed 2nd ballot draft complete Feb 22, 2009. • Team comments submitted updated draft. Feb 24, 2009 • Completed comment incorporation – March 10, 2009 Benefit of project • Alignment of assessment tools and methods to process based management QMS. • Shift audit focus to process, product, system performance and customer satisfaction. • Reduced variation of training and training expectations. Unresolved issues & decisions (if any): None – Sector rejection by the AAQG and member rejection by EAQG – Germany, triggered a revision to the document and 2nd ballot process. Schedule for 2nd ballot complete. Next steps (30-60-90 day look ahead): 1. 2. AAQG Briefing F2F Portland. March 10-12, 2009 IAQG Release of reballot. March 11, 2009 3. 4. 5. 6. 7. AAQG Release of 2nd ballot. March 20, 2009 AAQG Ballot cycle. March 20-April 20, 2009 EAQG Ballot cycle. March 20-May 20, 2009 APQG Ballot cycle. March 20-May 20, 2009 Target Release July 2009 Development Model for a Process-Based 9101 Process Approach Audit Records Evidence of Conformance Tools Process-Based Auditing Evidence of Effectiveness Effectiveness Assessment Record 1 2 Report Number: 3 4 Issued Date: Organization: Site: □ Core Process Assessed: □ Support Process Assessed: □ Foundation Process Assessed: 5 Core, Support & Management Processes 6 9100/9110/9120 Section Reference: 7 Support Process(es) Assessed: 8 Statement of Effectiveness: Inputs Output □The assessed core/support/foundation process has been determined to be effective. □The assessed core/support/foundation process has been determined to be ineffective. 9 How was process effectiveness determined (methodology)? 10 Objective Evidence, Observed Conditions, Data, and Comments to Support Effectiveness Determination 11 Assessment Team Leader 12 Signature and Date Signature indicates that all relevant requirements (including requirements for support processes identified in Block #7) have been assessed to determine effectiveness of the core or support process identified in Block #5. Effectiveness Focused 9101: Major Changes Review Introduction in the audit report of: an “Process Effectiveness Assessment Record” form (PEAR) which : State if for the audited process has been determined that – planned activities and results are realized – planned activities are realized but planned results are not realized – planned activities and results are not realized Give objective evidence, observed conditions, data, and comments to support effectiveness determination a process based “Audit Completeness Record” an “Objective Evidence Record” capturing quality manual and procedures with objective evidence of conformity 9101: What has not changed • Determining conformance to the standard. • Nonconformance discovered during the audit are to be documented and must be closed before certification. • Based on information collected during the assessment, the auditor will have to draw conclusions on conformance of the organization quality management system. Note: Conclusions on effectiveness are new 9101: Progress Status • Issue of a coordination draft : 30 June 2008 (Limit date for comments : 7 July 2008) • Review of the comments (8-11 July in Amsterdam) – • 9101 Formal Ballot 450 comments received, 233 accepted, 15 partially accepted 126 comments reviewed and disposition from ballot rejection (January 9-12, 2009 in Huntington Beach • • • • Publication of the 9101 Formal ballot : Sept./Oct 2008 Disposition on comments : end Nov. 2008 – Rejected Comments addressed – January 2009 Preparing for ballot release – March 2009 9101 Release Target Release – July, 2009 9101 Status Update • Ballot version published 26 September 2008 • Results of ballot: – APQAG : passed, with comments – AAQG: rejected, with comments – EAQG: • • • • • • France: passed, as is Germany: rejected, with comments Italy: passed, as is UK: passed, as is Switzerland: abstention, no experts Sweden (VAC): passed, with comments • 9101 team conclusion: address the comments and go for 2nd ballot Feedback summary 126 detailed comments received Major comments: – Checklist / record for objective evidence is missing (US, Germany, Sweden) – Structure of the document needs improvement (AAQG) – During OP assessment NO attention should be given to customer specific QMS requirements (Germany) – PEAR form and approach is too complex, e.g.‘all processes’ 9101 team proposals: – To add record for objective evidence – To restructure part of the standard: • All methodology / approach issues will be combined in one chapter • Recommended practices will be in a separate document (ARP, TR?) – Use of PEAR form limited to Product Realization Processes only Dispositions • 126 comments received • Team meeting in Huntington Beach 10-12 February, 2009 • Disposition statistics: – – – – – Accepted: 59 Partially accepted: 17 Rejected: 39 Deferred: 8 Noted: 3 AAQG P61 - IAQG 9101:2009 Revision March 10, 2009 Planned Vs. Actual with Slips Page 1 of 1 ID Name 2008 % Comp 2009 2010 2011 N D J F MAM J J A SO N D J F MAM J J A SO N D J FMAM J J A S O ND J FMAM J J 9101 Development and Rewrite 100% 1st Ballot Release 100% 2nd Ballot Release 69% IAQG Release of reballot 0% AAQG Release of 2nd ballot 0% EAQG Ballot cycle 0% APQG Ballot cycle 0% Ballot Call for Vote 0% Review/Dispostion Comment from Reballot (Possible F2F) 0% Publish 9101:2009 0% Implemenation of 9101 2% Current Plan Baseline Status 9101 Release Schedule Maximum 30 month Implementation from Publication 01/09 9100:2009 Publication 02/09 03/09 04/09 07/09 9101 Ballot start IAQG Phoenix New 9101 Publication Notes: • All changes in 2nd version are related to comments • No new requirements, except use of Objective Evidence Record as checklist Request: to reduce 2nd ballot in Europe to 2 months Questions Comments/Feedback to: email@example.com Back Up AS9101 Process Audit Flow From Pre-audit Planning Stage 1 1.0 Conduct Process and Business Research 4.0 Data Collection 8/20/08 – 9/3/08 3.0 Complete appendix # 6 Customer Quality Data 2.0 Conduct Planning Interview 5.0 Complete appendix # 1 Identify clauses and processes (not applicable for this audit) 6.0 Complete appendix # 5 Stage 1 Audit 7.0 Develop micro audit plan Due 9/3/08 Due 9/3/08 Stage 2 8.0 Conduct Interviews & follow Audit Plan Conclusions & Reporting Audit Lead Auditors 12.0 Complete appendix # 7 EAR Form 9.0 Complete appendix # 4 Summary 13.0 Convene team for Consolidation / roll up (at lead discretion) 10.0 Update appendix # 1 11.0 Complete appendix # 2 Write NCRs 14.0 Complete appendix # 3 Audit Report Beta Test Results – 1st Party View • Review and analysis of AS9101 Coordination Draft • Conclusion made by the auditors was that the approach is good. Our auditors are trained in the process approach and use the ISO 9001 Auditing Practices Group guidance documents. • All elements of a process audit were captured and defined but lacked sequencing of the activities. • Process flow was created. • Audit team formed and training on the process was conducted. AAQG Project #61 (IAQG REQ-009) Scope Amend 9101 Assessment standard in order to enhance its clarity and compatibility with process based management QMS (ISO 9001) and IAQG 9100 Standard update to be published mid 2009 AAQG Project #61 (IAQG REQ-009) Purpose of Revision • Focus a methodological approach of quality systems audits • Enhance clarity of standard to resolve issues on how to use the standard • Keep a standard report format • Consider all stakeholder needs • Ensure that the audit approach is more process oriented • Extend the scope of the audit process not only to determine conformity but to include the assessment of the organization’s ability to meet customer, regulatory and the organization’s own requirements Meeting 13-15 November 2007 Toulouse Summary of discussions/decisions • Scoring will no longer be required under the new 9101 • Audit Process Matrix list with future 9101 topics agreed • Inputs for new 9101 determined: • • • • • • Audit Process Matrix List ISO TC 176 APG material Stakeholder inputs Automotive IATF material for for auditing ISO/TS 16949 Requirements flow down chapter from SCMH Referenced documents determined: • • • • ISO/IEC 17021 Part 1 and 2 (if available) ISO 19011 IAQG 9104 IAQG 9100/9110/9120 • Relation with new 9101 structure and referenced documents ‘mapped’ • Detailed writing rules for drafting new 9101 were agreed • Writing tasks assigned to sub-teams • Schedule revised, now aligned with 9100 schedule Meeting 12-13 February 2008 Orlando Summary of discussions/decisions • Sub-team report out • Assembled working draft • Working Draft Review • Some requirements had direct relationship to AS9104 rewrite • Developed list of input for Berlin OPMT meeting • Team size is becoming a factor. • Agreed after Naples working session, team would be scoped down increase velocity of development. • Output of smaller team would come back to original team.