Juhani Anttila
Venture Knowledgist Quality Integration
Helsinki, Finland




Practical experiences of the use of the Malcolm Baldrige (MB) Model, the criteria and methodology of the American National Quality Award, are considered in this paper. As a case study the use of the methodology has been considered on the basis of experiences obtained within Sonera Corporation, the leading telecommunications operator in Finland, where performance excelennce models were nearly ten years as a major quality management tool for enhancing the overall business performance towards excellence. Although this paper considers Malcolm Baldrige model as the example the experiences are relevant also to those cases where some other performance excellence model, e.g. the European EFQM model, is used.

The case company, Sonera Corporation, and the development of its quality approach

Sonera Corporation was a leading provider of mobile and advanced telecommunications services. Sonera was growing as an operator, as well as a provider of transaction and content services in Finland and in selected international markets. The company also offered advanced data communication solutions to businesses, and fixed network voice services in Finland and neighboring markets. In 2003 Sonera was merged into Swedish Telia Corporation and continued as TeliaSonera's country organization TeliaSonera Finland Ltd.

Sonera's quality approach and its deployment have been developed over the several last decades and along with the development of company's overall business environments and requirements. The following stages and emphases are to be distinguished during the decades pertaining to quality development at Sonera:
- 1960: Technical specifications; ensuring problem-free functioning of the telephone network; reliability of technical equipment and systems
- 1970: General international standards (especially ITU Recommendations and IEC Standards) for telecommunications; standards-based quality of services; requirements for and control of the suppliers of telephone network equipment and systems
- 1980: Customer perception and satisfaction pertaining to the company's products (telephone and data communication services)
- 1990: Quality Management (QM) approach supporting business performance improvement
- 2000: Integrated QM with business management; producing benefits to and satisfaction of all stakeholders (interested parties); striving for excellence in business performance; responding to e-business challenges ("e-Quality" approach)

Figure 1. The overall business performance is based on business activity (processes and projects) and their results. For excellence one should compare performance with relevant references

The aim of QM approach at Sonera is to contribute for the company's strategic and operational performance goals. QM is a specialized expertise for enhancing effectiveness and efficiency of business management and leadership. Thus genuine realization of the QM practices takes place in real business activities both in the company's strategic leadership as well as in the operational realization of the company's services. In this approach all relevant stakeholders (interested parties) are also associated in accordance with their appropriate roles. This means a business-integrated quality approach.

The goal of the overall business performance and also the QM is business excellence (see figure 1).. Business excellence means a competitive balanced business performance including financial, customer-perception, business process, and innovativeness and learning perspectives When considering performance both activity and its results are taken into account. The objectives are reached through innovative management and leadership practices. In order to realize that in all parts of the company and at all levels of business and business management, an organization-wide management structure, a leadership infrastructure framework, has been defined. The framework model (see figure 2) was created at Sonera in early 1990s. This model covers all business activities in a natural and flexible manner, and it also defines responsibilities. This is also the framework to implement all QM-related means, e.g. including the Malcolm Baldrige methodology.

Figure 2: Sonera's QM realization model: The infrastructure framework of four levels of QM learning and competence within the corporation

Sonera's QM approach is well harmonized with the company's business values. On that basis also the quality policy has been defined. This quality policy articulates general intention and direction towards quality and includes following items:
- We always act so that the customer receives what he or she needs.
- We do what we promise.
- We improve our activities and their results continually so that they will become better and more effective and efficient.

The overall intention at Sonera has generally been to seek actively original sources of QM information and thus be able to avoid e.g. too extensive use of the services of external consultants. For example, the ISO 9000 standards have been utilized through participating right from the start in their drafting as Finland's national representative and expert in the international standardization committee ISO TC 176 for quality management and quality assurance.

Business excellence tool kit

A clear guiding ideas and principles concerning quality and QM as well as a comprehensive, company-wide realization model (see figure 2) for mobilizing the "ideas" are not enough for getting quality happen in a professional way. Practical means, tools, methods, etc., especially relevant management methodology, are necessary to get the approach concrete in practice. For this purpose, a collection of management tools has been created at Sonera. In addition to specialized quality management methodology, Sonera's "Business Excellence Tool Kit" includes also tools e.g. for finance, human resource, risk management, technology management, acquisitions and marketing. The most essential quality management tools include:
- the process management model
- the Malcolm Baldrige (MB) self-assessment procedure
- process auditing (the empirical assessment procedure of business process performance)
- the benchmarking procedure
- the Sonera Smart Scorecard procedure (developed from the balanced scorecard methodology)
- the project management model
- the problem solving and improvement procedures

These tools cover the needs of a comprehensive quality management. One of the principal needs is performance evaluation. The critical evaluation of the business performance and related strategic decisions on targets and actions are key measures in continually increasing the effectiveness and efficiency of the business performance. In order to evaluate comprehensively the overall situation and its development of the business performance, the quality award methodology is applied. Assessments based on quality award criteria have been used in Sonera as internal self-assessments by boards of directors in different business units since 1992. The assessments are based on the criteria of the American Malcolm Baldrige National Quality Award. Quality award criteria based assessments are applied for the whole business units. Thus, they are related to strategic business management. By utilizing general quality award criteria, a business is able to be placed on a "global map" of overall business performance. One can find the level at which
- the business performance itself is,
- the partners and competitors are, and
- the best Finnish, European, and international companies are.

Sonera's business units have also three times applied the Finnish National Quality Award in order to "calibrate" the internal self-assessments. However, even participation in the quality award competition is understood as a part of self-assessment to improve business performance.

Assessing business process performance constitutes a central part of the operational QM. The assessment procedure has been developed on the basis of ISO 9000 auditing principles (cf. Standard ISO 19011) and by combining ideas from business requirements and quality award criteria in it. In assessing process performance, the focus is on the critical evaluation of procedures applied in the processes and process results achieved through their implementation. Based on assessment results, Sonera has a corporate-widely used process performance indicator, with which one tracks how the performance of business processes has been developed.

Malcol Baldrige methodology for business excellence

Different quality awards models have been developed in many countries. The American Malcolm Baldrige Model is generally considered as matured and effective one. Many other national models are based on it. At Sonera this model was selected as a basis of a deep insight from a thorough investigation including training and benchmarking in the USA. The top management of the corporation made the final decision on the methodology and investments into its implementation.

The self-assessment activity started at Sonera in 1992. The initiating impact to the activity was a study made on the business performance measures and indicators and the overall performance management in the corporation and in its business units. One observed that the situation was not satisfactory. There were in use a lot of distinct performance indicators without any effective coordination and measurements unsystematically based on separately developed procedures. The situation was a quite natural result from the uncontrolled evolutionary development initiatives. The quality award methodology - learnt by the company's quality experts from the USA - was seen as an opportunity to get a comprehensive grip of the problem. From the original Malcolm Baldrige Model Sonera's own practical MB procedures and software tools for the self-assessment (tailored to Sonera's purposes by a Finnish company Finnevo Ltd) were developed.

MB criteria were very suitable when the company was just started orienting towards the competitive international markets including the USA. Benefits for the internal business communication were obvious, too, because with the MB approach it was possible to get an understanding of the overall business performance against a world-widely recognized score. With MB methodology also the quality management approach really got integrated with the key business management activities of he strategic management process.

MB criteria have the following important functions to strengthen organization's competitiveness:
- helping improve organizational performance practices, capabilities, and results
- facilitating communication and sharing performance information and best practices information among organizational units and partners
- serving as a working tool for understanding and managing performance and for guiding planning and opportunities for organizational learning

The award criteria used in self-assessments have produced highest benefit within a diagnostic framework. In this sense self-assessments differ from quality award assessments which are designed to produce scores to find the winners rather than make diagnoses.

The criteria are built upon a set of core values and concepts recognized in modern business organizations. The values and concepts are the embedded beliefs and behaviors found in high performing organizations. These values are the foundation for integrating key business requirements within a results-orientated assessment framework that creates a basis for a strategic action and feedback. The core values and concepts of the MB model - which were very relevant also for Sonera's business development situation - are:

- Visionary leadership
- Customer-driven excellence
- Organizational and personal learning
- Valuing employees and partners
- Agility
- Focus on the future
- Managing for innovation
- Management by fact
- Public responsibility and citizenship
- Focus on results and creating value
- Systems perspective

The MB model is based on the principle that in order for an organization to succeed, there is a number of key business enablers on which it should concentrate to achieve competitive goals. It also needs to measure its success through a number of key results areas. Thus, the core values and concepts are embodied in seven assessment categories of the MB criteria (see figure 3). These categories include 18 examination items, and these in turn contain a total of 29 areas to address. In the MB procedure, business performance facts are described responding to the questions of the areas to address, and the assessments (according to the scoring principles) are made by examination items. A major part of the examination items refer to the different approaches and their deployment for the business realization, whereas other part refers to performance results.

Figure 3. Malcolm Baldrige criteria structure

Malcol Baldrige criteria work well with ISO 9000 standards

In general, there are two recognized principal source materials that are useful for the development of the company's quality approach: ISO 9000:2000 standards and quality awards models. At Sonera both are used as general QM references. They support very well each others because they have the same basic aim, excellence of the business performance, and very similar structure and contents elements. However, they have also differences that originate from their backgrounds. While the standards reflect a large international consensus for harmonization of the approaches for quality management and quality assurance, the quality awards models are based on the development of a national competitiveness towards excellence. In the standards the views of quality experts are emphasized, and the awards models reflect especially the business management views. It is just due to both the similarities and the differences that their simultaneous use is beneficial for the company. Thus, it is very important for the development of the company-wide quality approach to understand the effective relationship of these important references of the professional quality management. Then also possible conflicts between them can be avoided.
The award model does not give any recipes for the development. The award criteria are useful in the strategic performance assessment and the standards in searching standard guidance for the development of business performance after assessments. In fact, the self-assessment activity - including the quality awards methodology - is also an important element of the ISO 9004:2000 standard. The standard even includes a simplified method for self-assessment that is based on the quality awards models. However, advanced organizations can go directly to using awards models.

At Sonera self-assessments started directly using the Malcolm Baldrige Model but later through learning by action also own insight was born and own practices developed. In that context also particular material was created including small pocket-books "Good Better Best" and "ISO 9000 for the creative leader" for company-dedicated insight of both the Malcolm Baldige quality award model and ISO 9000:2000 standards.

What is self-assessment all about?

Self-assessment is a way of looking at how well a business unit is performing. It enables a unit to look across all its activities, set a stake in the ground for its performance to date and determine what it now needs to do to make improvements in that performance.

The key enablers by which the top management judge the unit's performance are:
- How well the unit is led
- How well its human resources are managed
- How far its values, vision and strategy are developed and implemented by its leaders and realized by people
- How well it manages its resources and developes and manages its business processes
- How well information and knowledge are managed for the needs of business management

The key results areas by which the criteria then measure successes are:
- How far it satisfies its customers
- How well-motivated and committed its work force is
- How the local and national community outside the unit views its activities in terms of its contribution to society
- How its key financial and market-place results as well as internal business indicators (the business "fitness" indicators) have been developed

Basically MB criteria measure how well all the stakeholders 'requirements' are being met.

Using the criteria a business unit can score up to 1000 points on their performance. In scoring there is considered both approach (i.e. how to do things) and the results thereof. The criteria are divided into 18 assessment items that are all interlinked through the business logic (i.e. cause-effect relationships) (see table 2). What is significant in the score? A very important score (directly 170 points) is for customer relationship approach and satisfaction results. The best units focus on meeting all their customers' needs by providing them with a product that delights them. Practices for human resource management and people satisfaction are also seen as most important area with 165 points. Happy people serve their customers best and also do business best! This is closely followed by the measure of financial business results at possible 125 points, and process approach with 85 points and internal effectiveness results 120 points. In fact, achieving and exceeding business goals and targets is the most recognized result area in Malcolm Baldrige criteria. All these then are seen to be the most important measures of how well the unit is performing. But a unit will achieve little in these areas without effective leadership (a possible 120 points), and strategic planning (85 points). So far the best companies in the world are scoring around 800 points. A score of 500 points is already very good.

Practices of self-assessment

There are several ways in which self-assessments have been carried out at Sonera. The most usual way is that the top management team of a unit holds a self-assessment workshop going through all of the 18 assessment items of the criteria. To get maximal benefits from the assessment, written descriptions of strengths and weaknesses are at first collated per each item, and then the recorded facts are assessed by the team using the scoring principles. Also a Web-based software that was specially developed for Sonera is used to support the assessment. As a general rule self-assessment has been shown to work well when an unit:
1. Has clearly defined customer-segments
2. Knows well its key suppliers
3. Has clear set of products
4. Has consistently managed business and support processes
5. Has measured its results, and time-series of data is available
6. Is run effectively by the management team

In the early stages of the self-assessment the scoring should not be emphasized too much. There is a danger that the assessment is seen as a number-game. The quality award self-assessment is not, however, any mathematics but a systematic and critical business management task enhancing satisfaction of all interested parties (stakeholders) and company's own business effectiveness and efficiency. Finding strengths and weaknesses is in practice much more important than the scores.

The assessment as a whole consists of the following tasks (see figure 4):
- Acquisition of data for the questions of the examination items and preparing the written descriptions of strengths and weaknesses and optional informative issues
- Classifying the facts according their significance based on the purpose and goals of the business
- Assessing (scoring) the facts descriptions by examination items and individually by the members of the management team
- Consensus meeting by the management team
- Conforming the results (final statements of the strengths and weaknesses, and the scoring)
- Diagnosis of the results, and decisions and plans for the improvement actions

Preliminary considerations necessary for a self-assessment include:
- Identification of the organizational unit to evaluated (corporate, a whole business, a function)
- Preparing the general business profile (understanding what is relevant and important to the business unit)
- Embody clearly the business activities to be evaluated (strategic management, operations including business/work units/functions, business processes)
- Organizing the self-assessment (management decision on purpose and objectives, involvement of the management team, facilitation for a methodology mastery - CEO or quality manager, data acquisition and item descriptions, assessment time schedule)

Figure 4. Malcolm Baldrige self-assessment as a whole

Before starting a particular assessment, a general organizational business profile description is prepared. It is a good basis for the further actions and decisions during the assessment. Malcolm Baldrige criteria give good guidance for defining the profile. It consists of general descriptions of business environments, relationships with customers, suppliers, and other partners, and key strategic challenges, and practice for performance improvement. Another useful preliminary task before the assessment is to define company's current competitive edges. These are those issues that enable one to succeed repeatedly better than the competitor when utilized skillfully, regardless of any random factors. Competitive edges are superior when changing the situation would require significant amounts of resources, time, fundamental structural changes, another operating environment, renewal of technology, etc. from the competitors. Most competitive edges have to do with size, position, unique reciprocal relationships, special know-how, flexibility, speed, linkages, outstanding effectiveness and efficiency, etc. Competitive edges do not last forever. Excellence in current markets, with respect to current customers and in current product groups do not necessarily imply even mediocre competitive edges in the case of new customer groups and when introducing new products into the (new) market(s). Max. two most promising challenges should noted for the competitive edges pertaining to implementing the strategy of the organizational unit in focus.

For description purposes all the assessment questions of the criteria should be understood as "How" questions. In order to understand the hierarchy of the questions, at Sonera a particular hierarchy-navigation methodology was developed. The assessment criteria and scoring guidelines have to be understood in the light of the unit's own business requirements (cf. business profile). This calls for an interpretation as to how issues in the criteria are relevant in the light of the unit's business requirements and how such issues ought to be interpreted. The interpretation becomes clear when one asks oneself "should we also?" and, if so, "do our business requirements require something special regarding the approach or its deployment?" Also the business results must be relevant from the unit's business viewpoints.

The fact-description technique requires methodological decisions. Very short descriptions are fast, but they are not very communicative outside the assessment team - even there may be difficulties within the team. Also scoring is unsure and subjective when based on inadequate descriptions.

Fact-descriptions for the assessment implies that objective evidence is searched regarding company's strengths and weaknesses relating to the assessment items of the criteria. At Sonera there are quite rigorous definitions for the both concepts. Strengths are those facts that distinguish you as a success from others. Unique strengths are those singular issues which your most important competitive edges are based upon or on the basis of which you are able to meet new challenges exceptionally competitively. With the strengths one differs from others as a success when one thinks about competition at the levels of the line of business, organization, and product. Weaknesses are the hindrances to utilizing strengths in a superior manner. Competitiveness - superior competitiveness in particular - can't be created through patching up weaknesses. Success is based upon exceptionally skillful utilization of unique strengths. The question here does not, then, relate to any weaknesses whatsoever, but rather to those weaknesses which have relevant significance with respect to the competitive utilization of the strengths. In addition to the strength and weakness statements it may be useful also to note some informative issues regarding particular business activities and results. These may be facts that do not seem to be strengths nor weaknesses, i.e. they do not seem to be relevant at all for performance assessment purposes but may be noted for possible future needs.

Causal relations of the business management issues (e.g. strategic plans -> process operations -> business results) should be clearly understood in the business facts descriptions. These relations become significant particularly when scoring is above the level of 400 points. Realizing causal relations is promoted e.g. by:
- The balanced scorecard methodology
- Clarity of the organization
- Systematic process approach

Among the assessment team, different conceptions of one and the same issue provide an opportunity to learn. When proceeding appropriately in a self-assessment the outcome is either genuine consensus or a shared view of the diversity of views. Differences may concern relevant facts or scores. A true consensus on descriptions is most important. A consensus reached through a good interaction, an "argumentated and unauthoritarian discourse", creates a strong commitment to the preconditions of effective development. The factors that have a positive influence in achieving consensus include:
- Clear descriptions of strengths and weaknesses
- Knowledge of causal relations between business facts
- Common understanding of criteria and scoring principles
- Common and clear understanding of the business profile and requirements

The consensus procedure is a central and challenging part of all self-assessments. If there is no time for it, something substantial is lost. Reaching consensus is a difficult art. In a self-assessment consensus can be sought either through the individual work phase or directly in a group. Before the group phase starts the facilitator should recognize whether there are differing views and potential conflicts. When necessary he or she should select the correct conflict strategy. An inappropriate strategy may badly distort the assessment. The main options of the strategy are: covering up, pacification, heightening of awareness, and aggravation. In most cases the heightening of awareness is the best strategy.

Who should do the assessments? Roughly there are three possibilities:
- Own management team (preferable)
- External expert team (not suitable nor preferable for self-assessment)
- Partly own and partly external team (external persons should be in the role of facilitator)

Preferable there should be in the assessment team at least four persons who know the assessment methodology. The highest business officer, e.g. the CEO, should always be one of the assessors in self-assessments. It should never be trusted only in one single expert in the assessment. The possible external assessments, e.g. the assessments in context of applying the national quality award, should be reviewed and interpreted by the business leaders for necessary actions. Thus, those assessments are understood as a part of the own self-assessment. The assessments should be carried out as an assessment process (see figure 5).

Figure 5. Malcolm Baldrige self-assessment is carried out as a process

Professional Malcolm Baldrige scoring is fairly accurate (about 10 %-units), i.e. the same result may be obtained by different competent assessors. In internal self-assessments, however, differences may arise from the reason that the attending persons (although they were leaders of the same business unit) often have different tacit knowledge of the performance item). Then the assessment is not based is not only based on written descriptions of the facts but also on their personal opinions.

Figure 6. Scoring is a multi-facet task

Due to comparability, it is essential to use internationally recognized criteria and scoring principles also in internal self-assessments. Comparability can be ensured by:
- Following the original Malcolm Baldrige Model
- Having clear written descriptions of strengths and weaknesses as the basis for assessments
- Applying national quality awards (for calibration)
- Attending national quality awards competitions as assessors (enhancing assessment skills)
- Training own assessors and business leaders
- Using common model for fact-descriptions and assessment methodology, and supporting information technology

Self-assessment using a quality award model is not any easy task (see figure 6). However, the assessment technique is quite simple and easy to learn. The difficulty is originated from the substance itself. Critical understanding of a complex business performance requires competent resources and initiativity and commitment of the top management. Difficult issues are the understanding the multi-dimensional assessment entirety and the individual criteria items in terms of the own company.

Always after self-assessments, the management team should consider performance improvement actions as a part of the business strategies. ISO 9000:2000 standard gives good general guidance for the improvement methodology both regarding strategic breakthrough improvements and step continual improvements. It is impossible to reach above the level of 500 points without a systematic improvement practice in use.

Benefits of the self-assessment

One can find both internal and external benefits from using quality awards methodology and self-assessments for business performance improvement. Internal benefits include:
- Systematic and critical consideration of the business reality (examination items of the criteria as an agenda)
- Objective reference for systematic development of a company's business performance (strengths/weaknesses and score)
- Useful material for internal business communication
- Effective integration of QM with business management
- Opportunities for internal best practices and benchmarking

Correspondingly external benefits include:
- Respected criterion (practical and internationally recognized non-commercial basis, possibility to international and interbranch comparisons)
- Benchmarking
- Positiveness in public communication, and positive impact for a public promotion

Figure 7. Enhancing business performance towards excellence may happen only systematic assessments and improvement activities, i.e. PDCA cycles


The business leaders and employees have felt self-assessments that have based on the quality award methodology as positive and interesting learning experiences. The assessments have easily created development dialogues genuinely participated also by the top management. Systematic and objective self-assessment practice has created in the organization generally critical and analytical thinking. Self-assessment - especially finding, understanding, and describing real strengths of the business performance - has been a demanding management task. Based on many years' experience, one does know any better methodology for this task than a quality award model, especially the Malcolm Baldrige Criteria.

Units that have used the self-assessment methodology show a commitment and enthusiasm to provide continually improving service to their customers and in the same time also meet the needs of the other stakeholders. At Sonera the cycle of MB assessment and improvement continues to enhance our competitive position (see figure 7).

From the experiences at Sonera one can draw conclusions that self-assessments using the Malcolm Baldrige or any other quality awards criteria based methodology can be carried out more effectively and efficiently if:
- the company's management have a clear understanding of the basic QM/TQM/business excellence concepts
- the company have its own QM/TQM/business excellence model with Malcolm Baldrige as a tool
- the company should have widely-applied business process approach for strategic management and operational business management
- the self-assessment approach is based on the company's business needs and be integrated with business management
- the self-assessment approach is launched step-by-step
- the company have reasonably own expertise and resources for QM related development
- the self-assessments are done by own resources (by company's management team, and quality experts as facilitators)
- ISO 9000 standards methodology, e.g. auditing, and quality award criteria are used simultaneously to support each others
- also other recognized quality management tools, especially Balanced Scorecard and Hoshin Kanri, are used with Malcolm Baldrige model
- the self-assessment approach is being developed continually

The last ten years in Sonera have been a tough period, but the professional quality management efforts are definitely paying off. There have been great strides in its products and partnership with its customers and the community. The Finnish National Quality Award has been applied and finalist position achieved but still it is to challenge to the world class business excellence, i.e. a score around 800 points as measured by the MB criteria. Using self-assessment across the business units is helping the corporate continue to raise standards in every aspect of the business, thus enabling achieve the vision and strategy to became a most successful and competitive worldwide telecommunication business.


1. Anttila J., Vakkuri J.: Good Better Best, Sonera Corporation, Helsinki 2000
2. Anttila J., Vakkuri J.: ISO 9000 for the creative leader, Sonera Corporation, Helsinki 2001
3. www.quality.nist.gov: Internet pages of the Malcolm Baldrige Criteria
4. International Organization for Standardization: ISO 9004:2000. Quality management systems. Guidelines for performance improvement, ISO, Geneve 200


[This material has been presented in different forms in differerent seminars or conferences, e.g. in Mumbai, India 2002]