1.5.1 The IRS Balanced Measurement System: A New Approach to Measuring Organizational Performance Handbook

1.5.1.1 (10-01-2000)
Introduction

  1. This Section provides an overview of the IRS Balanced Measurement System and outlines how balanced measures, one of the five levers of change for modernizing the IRS, are used to support a new approach to measuring organizational performance. Because it is not possible to prescribe acceptable actions for every situation, this section should be viewed as providing general guidance and direction on the appropriate use and application of balanced measures.
  2. In determining the appropriateness of a specific course of action involving performance measurement, managers and employees are encouraged to exercise sound judgment consistent with and in support of the overriding principles described herein.

1.5.1.2 (10-01-2000)
Overview

  1. The IRS Balanced Measurement System has been developed as part of the effort to modernize the IRS and to reflect the Service’s priorities, as articulated in the IRS’s mission and strategic goals. To help ensure balance, each of the three components of balanced measures - customer satisfaction, employee satisfaction, and business results - will be carefully considered by the IRS when setting organizational objectives, establishing goals, assessing progress and results, and evaluating individual performance.
  2. The IRS will use balanced measures at both the strategic level and the operational level to measure organizational performance. At the individual level, critical elements and critical performance expectations that support and align with the mission and balanced measures approach will be the basis by which employees are evaluated.

1.5.1.3 (10-01-2000)
Balanced Measures Quick Reference

  1. The following excerpts summarize key points of this section.
    1. To help ensure balance, each of the three components of balanced measures - customer satisfaction, employee satisfaction, and business results - must be carefully considered when setting organizational objectives, establishing goals, assessing progress and results, and evaluating individual performance.
    2. Under the Balanced Measurement System, the approved set of balanced measures is the primary means for assessing organizational performance. Other information and data can be used for purposes such as workload planning, analysis ( “getting behind the numbers” ), or strategy selection.
    3. Goals for quantity measures must always be discussed in conjunction with customer satisfaction, employee satisfaction, and quality goals.
    4. Goals for any measure should be set based on a review of the previous year results, the anticipated mix of resources available, the linkage to organizational priorities and initiatives, planned process improvement or system enhancements, and an assessment of existing and emerging trends, issues and problems.
    5. In any instance when numeric organizational goals are shared and discussed, caution must be exercised to ensure that any such discussion does not imply or suggest numeric goals for an individual.
    6. The performance of any one unit should not be used as a standard by which the performance of any other unit would be evaluated due to differences that exist in the types of taxpayers served, employee skill levels, specific tax issues being worked, and other factors.
    7. The primary focus of organizational reviews should be on the actions taken and related accomplishments, not numeric results.
    8. The numeric results achieved with any of the balanced measures will never directly equate to the evaluation of an individual.
    9. Because measures reported by the IRS externally are subject to extensive evaluation and review, an especially rigorous set of standards should be met for this set of measures.
    10. In determining the appropriateness of a specific course of action involving performance measurement, managers and employees are encouraged to exercise sound judgment consistent with Regulation 801 and in support of the Balanced Measurement System.

1.5.1.4 (10-01-2000)
What is the IRS Balanced Measurement System?

  1. The IRS Balanced Measurement System provides a means to:
    1. Articulate organizational priorities and define what we need to focus upon as an organization.
    2. Guide and motivate performance and establish a linkage between performance goals and the organizational objectives.
    3. Obtain feedback that will help us ascertain how well we are doing in meeting customer and stakeholder expectations and identify areas for improvement.
    4. Assess overall program effectiveness and communicate results.
  2. The elements of the Balanced Measurement System are Customer Satisfaction, Employee Satisfaction, and Business Results, with business results being comprised of measures of Quality and Quantity.
  3. Each element represents an important aspect of the organization’s goals and each is of equal importance in carrying out the Service’s programs and functions. Because some of these elements do not change as rapidly as others or require more time for data collection, the frequency of measures data availability may vary across the three elements. However, such differences in the frequency of data availability do not reflect differences in priority.

1.5.1.5 (10-01-2000)
The Goals of the Balanced Measures Elements

  1. Goals of the Balanced Measures Elements.
    1. The goal of the Customer Satisfaction element is to provide accurate and professional services to internal and external customers in a courteous, timely manner.
    2. The goal of the Employee Satisfaction element is to create an enabling work environment for employees by providing quality leadership, adequate training, and effective support services.
    3. The goal of the Business Results element is to generate a productive quantity of work in a quality manner and to provide meaningful outreach to all customers.
  2. Balanced Measures are the measures used by the IRS to assess organizational performance at both the strategic level and the operational level.
  3. At the strategic level, such measures will be used to assess our overall progress in delivering on the mission and three strategic goals. Strategic measures will apply to the organization as a whole and to each of the major operating units in the modernized IRS.
  4. At the operational management level, measures are used to assess the effectiveness of specific programs (e.g., submission processing, filing and account services management, field collection, field exam, etc.).Figure 1.5.1-1
    BALANCED MEASURES
    Customer Satisfaction Employee Satisfaction Business Results
    Quality Quantity
    Purpose/Goal To serve customer professionally To provide an enabling work environment To do quality work To generate a productive amount of work

1.5.1.6 (10-01-2000)
The Shift to Balanced Measures

  1. The Balanced Measurement System was developed as part of the effort to modernize the IRS and to reflect the Service’s priorities, as articulated in the mission statement. This approach to measurement shifts the focus of individuals and the organization away from achieving a specific target or number to achieving the overall mission and strategic goals of the IRS.
  2. Under the Balanced Measurement System, the IRS still collects and uses performance results ( “numbers” ), but it uses these results much differently than it has in the past. Experience has shown that successful organizations cannot be managed by numbers alone. The numbers are only an indicator of past performance and, when considered by themselves, do not provide a complete picture of what is happening throughout the organization.
  3. In some instances, the IRS’ historic emphasis on achieving numeric targets failed to adequately consider the impact on quality case work, fair treatment of taxpayers, and employee satisfaction. The challenge given to the IRS through the Restructuring and Reform Act of 1998 was to develop an improved method of measuring performance that protects taxpayer rights, fosters quality service, and considers the impact on employees. The Balanced Measurement System is the IRS’ attempt to meet that challenge.

1.5.1.7 (10-01-2000)
Balanced Measures and the IRS Management Model

  1. The Service’s management model represents a clear approach to managing our business and is directly linked to our mission, goals, and the Balanced Measurement System. The model is comprised of four elements: Plan, Do, Review, and Revise.Figure 1.5.1-2
    This image is too large to be displayed in the current screen. Please click the link to view the image.
  2. We are not managing programs to achieve numbers; we are managing processes and people to achieve the IRS mission. Hence, using the Management Model, managers should collaborate with employees in developing plans, reviewing progress, and revising action plans.

1.5.1.8 (10-01-2000)
Using Balanced Measures

  1. The Balanced Measurement System enables the Service to:
    1. Measure how an organizational unit has performed relative to its past performance.
    2. Identify areas that need action to help improve performance, taking all the measures – Customer Satisfaction, Employee Satisfaction, and Business Results – into consideration.
    3. Align and support various review processes so that there is communication throughout the organization.
    4. Begin a dialogue between managers and employees that is focused on discovering and positively impacting the factors that influence performance.
    5. Provide input to managerial performance appraisals, not serve as a direct evaluative tool.
  2. For an example of the Balanced Measures see Exhibit 1.5.1–4.

1.5.1.9 (10-01-2000)
Using Performance Information

  1. The IRS collects a great deal of additional information about programs and services, some of which had been used as performance measures in the past. Under the Balanced Measurement System, the approved set of balanced measures will be the primary means for assessing organizational performance. Other information and data will be used for purposes such as workload planning, analysis ( “getting behind the numbers” ), or strategy selection.

1.5.1.9.1 (10-01-2000)
Diagnostic Tools

  1. Some indicators, hereafter referred to as “diagnostic tools” can be used to help analyze the factors that affect changes in the balanced performance measures in order to “get behind the numbers” and identify improvement opportunities. The use of diagnostic tools provides a mechanism to study the factors that influence performance and encourages dialogue about specific actions that can be taken by managers to improve customer satisfaction, employee satisfaction, and business results. Improvement projections may be established for diagnostic tools, but only in direct support of the overlying balanced measure. Like balanced measures, such projections will not be used in individual performance evaluations.

    Example:

    “Tax law accuracy” is a quality balanced measure. Diagnostic tools for this measure include each of the individual factors used to assess the accuracy of tax law information provided by assistors to taxpayers. In analyzing the factors that may be causing a decline in the overall score for tax law accuracy, it is found that one particular review factor is not being met 45% of the time. The organizational unit, in an effort to improve the accuracy of tax law information provided to taxpayers, establishes an improvement projection for that factor that is tied to an improvement in the overall score for tax law accuracy.

  2. Diagnostic tools can include any type of data that is helpful in understanding what influences and impacts the balanced measures. In some cases, data used as a diagnostic tool for one organizational unit (i.e. cycle time) may be used as a balanced measure for a different organizational unit and vice versa as long as the measure conforms to the guidelines and restrictions set forth in section 2.
  3. Examples of Diagnostic Tools and examples on how to apply those tools in conducting analysis are available in the Reference Guide for Diagnostic Tools and the Management Analysis Resource Guide, both of which can be located on the IRWeb at: http://www.hq.irs.gov/Bm/measures.htm.
  4. Diagnostic Tool Examples
    1. Results for individual questions on customer satisfaction surveys
    2. Results for individual questions on employee satisfaction surveys
    3. Cycle Time
    4. Employee experience/training/skill levels (i.e. hours of training per employee, work force mix, average educational attainment of newly hired employees)
    5. External factors (i.e. tax law, macroeconomic cycles)
    6. Employee absenteeism, turnover rates
    7. Physical resources
    8. Work policies
    9. Return closures per unit of effort
    10. Inventory level
    11. Survey rate
    12. Examined return disposition mix
    13. Workload mix
    14. No change rate
    15. Staffing resources (FTE appropriated, FTE realized, resource utilization)
    16. Results for individual quality standards/elements
    17. Wait time/transaction time
    18. Cost information
    19. Records of Tax Enforcement Results (ROTERs)
    20. Workload indicators such as returns filed.

      Note:

      No goals or targets will be set for ROTERs.

1.5.1.9.2 (10-01-2000)
Using Diagnostic Tools

  1. Do not use diagnostic tools as organizational performance measures or to measure individual performance.
  2. Improvement projections or goals, when set for diagnostic tools, must be established in support of the overarching balanced measure.
  3. Comparisons with other units may be appropriate for conducting analysis, exploring best practices or seeking process enhancements to support improvement of the overarching balanced measure(s).
  4. Do not use diagnostic tools for comparisons with other units when evaluating organizational performance.
  5. Use diagnostic tools to understand underlying factors that cause changes in the Balanced Measures and to identify “root causes.”

1.5.1.9.3 (10-01-2000)
Workload Indicators

  1. The IRS uses data referred to as “workload indicators” for purposes of resource planning during the development of program plans and budgets. Workload indicators project expected levels of activity for an organizational unit or program and are necessary in order to identify resource needs and justify the IRS’s resource requests to external stakeholders (e.g., Treasury, OMB and Congress). Examples include number of refunds issued, number of walk-in customers, number of returns, etc.
  2. Workload indicators will not be used in individual performance evaluations. They are included in organizational performance assessments only for purposes of comparing actual workload to the planned amount for use in making future projections and estimating related resource requirements.
  3. Workload indicators are typically not Balanced Measures, but they can be diagnostic tools.

    Example:

    Total Number of Calls Received is a workload indicator that is used to project staffing and resource requirements for the Toll Free program. It is also a diagnostic tool for the Toll Free Level of Service Balanced Measure as it can be used to help understand changes in the overall Balanced Measure (i.e., a significant increase in the number of calls received can help explain a decline in the Level of Service).

1.5.1.10 (10-01-2000)
Balanced Measures and the IRS’s New Program Structure

  1. In FY 2000, the IRS introduced a new Strategic Planning and Budgeting process that governs the development of Servicewide and Operating Unit Strategy and Program Plans (See Exhibit 1.5.1–5).
  2. Plans are created around a new program structure that is based in part on a redefinition of our business into three-core taxpayer-interaction categories:
    • pre-filing taxpayer assistance and education,
    • filing and account services, and
    • post-filing compliance.
  3. Each category is comprised of a number of specific programs. Following is a description of how the Balanced Measures framework applies to the new program structure.

1.5.1.10.1 (10-01-2000)
Measuring Employee Satisfaction in the IRS’s New Program Structure

  1. Employee satisfaction is measured for organizational units, sub-units, and work groups. It is typically not measurable for programs because employee time is often allocated to more than one program and many programs are spread across organizational units.
  2. The regulatory intent of the Employee Satisfaction component of balanced measures is to ensure that employee satisfaction issues and needs are considered and addressed in organizational planning, budgeting, and review activities (e.g., strategies to improve business results must consider and address, when applicable, the impact on employee satisfaction).
  3. Organizational plans should include specific strategies and initiatives aimed at improving employee satisfaction.

1.5.1.10.2 (10-01-2000)
Measuring Customer Satisfaction in the IRS’s New Program Structure

  1. Information to measure customer satisfaction is gathered from a statistically valid sample of the customers served by a particular operating unit. This is accomplished through the use of strategic surveys for the organizational unit overall and transactional surveys for its major programs.

    Note:

    Major is a qualifier intended to refer to programs that:

    1. impact a significant proportion of the customers served by the business unit, and/or
    2. utilize a significant proportion of the total resources (FTE and dollars) available within the business unit
  2. While customer feedback should be obtained, where possible, for all programs, a survey instrument or satisfaction score may not be possible or meaningful for every program.
  3. The intent of the Customer Satisfaction component of balanced measures is to ensure that customer satisfaction issues and needs are considered and addressed in organizational planning, budgeting and review activities (e.g., strategies to improve business results must consider and address, when applicable, the impact on customer satisfaction.)
  4. Organizational plans should include specific strategies and initiative aimed at improving customer satisfaction.

1.5.1.10.3 (10-01-2000)
Measuring Business Results in the IRS’s New Program Structure

  1. At the operational level, business results are assessed through measures of quality and quantity. Quantity measures consist of outcome-neutral production and resource data such as number of cases closed, work items completed, hours expended, etc. Quality measures consist of data such as case/call review data, accuracy and timeliness. The intent of the business results measures is to ensure that a productive quantity of work is generated in a quality manner.
  2. A regulatory limitation on the use of quantity measures requires that goals or targets not be set for such measures except in conjunction with goals based also upon customer satisfaction, employee satisfaction, and quality measures. Therefore, in the establishment of organizational goals for business results quantity measures, the organization must consider and address the impact of those goals on the customer satisfaction, employee satisfaction and quality goals of the organizational unit.
  3. Under the IRS’ new program structure, a program business results quantity measure must have a corresponding program results quality measure(s) and organizational customer satisfaction and employee satisfaction data available as described above in order to establish quantity measure goals. Absent meeting this requirement, projections may be set for quantity measures at a Servicewide level for purposes of workload planning, resource allocation, strategy selection, and financial planning. However, such projections cannot be used for evaluative purposes.
  4. As a rule: goals for quantity measures must always be discussed in conjunction with customer satisfaction, employee satisfaction, and quality goals.

1.5.1.11 (10-01-2000)
Integrating Balanced Measures into Measures-Related Processes

  1. The Balanced Measurement System is about changing the way the organization uses measures; it is not about eliminating the use of measures. Without measures it would be very difficult to effectively manage the Service or gauge progress in meeting our tax administration responsibilities.
  2. The work performed by the IRS has the potential to substantially impact the lives of citizens as well as their trust in government. As government employees entrusted with ensuring the public good, it is the responsibility of each employee to make certain that taxpayer rights are upheld and protected.
  3. For these reasons, the IRS must exercise great care and caution in how it uses measures. It cannot support a business and management approach focused only on the achievement of certain numbers and targets. Past behaviors and practices that may have contributed to such an approach have to change.
  4. The Balanced Measurement System has been designed to help employees, both managerial and non-managerial, actively engage in an approach to management and measurement at the IRS that is focused on identifying and taking appropriate actions to improve performance and on diagnosing the underlying factors that have influenced organizational outcomes. Following is an explanation of the use of measures under the Balanced Measurement System.

    Note:

    Portions of the guidance in this section do not apply to the “Pipeline” area of Submission Processing because of the nature of the work performed in that component of the organization. Policies regarding individual quality/quantity information, establishment and sharing of goals and data, and performance evaluation procedures based on existing organizational and work agreements in Submission Processing take precedence over this section.

1.5.1.11.1 (10-01-2000)
Setting Goals/Plan Development

  1. The establishment of goals and the development of plans must consider and address all elements of the Balanced Measurement System - Customer Satisfaction, Employee Satisfaction, and Business Results. Performance goals are estimates of results expected for a given period of time. They are included in plans as indicators of what it is an organizational unit hopes to achieve.
  2. There are two types of goals an organization can use to communicate priorities and guide performance; qualitative* and quantitative. Qualitative goals are general in nature and suggest a desired direction but do not establish a numeric target, i.e. “Improve Customer Satisfaction.” Quantitative goals, hereafter referred to as numeric goals, are specific and do establish a numeric target, (i.e. “Improve Customer Satisfaction from 70% to 80%” ).

    Note:

    *The term “qualitative” is not the same as the Quality measure in Business Results. “Qualitative” describes the type of goal, not the type of measure. For example, a qualitative goal for the Business Results Quality measure might be “improve the accuracy of tax law information provided to taxpayers.” A quantitative/numeric goal for the Business Results Quality measure might be, “Improve the Tax Law Accuracy Rate from 85% to 90%.”

  3. Qualitative and numeric goals will be set for the balanced measures at the Servicewide and operating unit level to satisfy requirements of law such as the Government Performance and Results Act. They will be used by the Service to report on agency progress in delivering its tax administration responsibilities. Additional numeric goals can be set within operating units to the level for which data for all components of balanced measures is available. Qualitative goals can be established at all levels of the organization to support organizational strategies and plans.
  4. Goals for any measure should be set based on a review of previous year results, the anticipated mix of resources available, the linkage to organizational priorities and initiatives, and an assessment of existing and emerging trends, issues and problems.
  5. The following criteria may further help in the establishment of targets. Targets should:
    1. be developed by those who will be held accountable,
    2. include input from customers and stakeholders,
    3. be derived from benchmarking, where available,
    4. represent realistic expectations toward meeting objectives, and
    5. be adjusted based on experience and expectations.

1.5.1.11.2 (10-01-2000)
Communicating Goals

  1. The IRS needs to exercise caution in how numeric goals are shared within the organization in order to avoid the numbers-focused pressures that were felt under previous measurement systems. Any sharing of goals must be done in a way that does not encourage the competitive environment that existed previously among some organizational units whose efforts were directed at achieving numeric targets without carefully considering the impact on all elements of the Balanced Measurement System - Customer Satisfaction, Employee Satisfaction, and Business Results.
  2. Operating Divisions are to deploy operational measures to the field level for FY 2002. Servicewide and Operating Unit goals will be included in organizational documents that are distributed broadly both within and outside the organization such as the budget submission, the annual performance plan, and the strategy and program plans. These documents and the goals included therein can be shared and discussed with both managerial and non-managerial employees at the discretion of management within each operating unit. Again, as a general rule: goals for quantity measures must always be discussed in conjunction with customer satisfaction, employee satisfaction, and quality goals. And, in any instance when numeric organizational goals are shared and discussed, caution must be exercised to ensure that any such discussion does not imply or suggest numeric goals for an individual.
  3. Prior to sharing or discussing organizational goals, the following questions can be considered to help determine the need for and appropriateness of the communication. In reaching a final decision, exercise careful judgment. Consider documenting the answers to the following questions as appropriate to help weigh the pros and cons to sharing goals and to help explain and assess the reasonableness of the decision that is reached.
    1. What is the business reason for communicating the Goal(s)?
    2. What is the business risk of not providing the Goal(s)?
    3. What is the potential undesirable outcome that could come from the misuse of the Goal(s), and how that outcome can be minimized?
    4. What is the risk that the intended recipient(s) would reasonably believe that the communication suggested the apportionment of an organizational goal into an individual quota or goal?
  4. Regarding the last element, consider:
    1. The degree of organizational knowledge and understanding of the intended recipient(s).
    2. The organizational climate at the time and place of the communication.
    3. The context in which the communication is to be made.
    4. Any guidance on how the Goal can or cannot be used.
    5. The manner in which the communication is delivered.
    6. The expectation of follow-up with respect to the Goal and the nature of the expected follow-up.
    7. The probable perception of the communication of the Goal The probable public perception of the communication of the Goal.
    8. The probable public perception of the communication of the Goal.

1.5.1.11.3 (10-01-2000)
Sharing Data

  1. Under the Balanced Measurement System, an Organizational unit may share balanced measures and other non-ROTER data with other units at the same level. The reasons for the Service’s position in this regard are threefold. First, 26 C.F.R. Part 801, which defines how the Balanced Measures must be administered, does not restrict the sharing of balanced measures data. Second, the Service has determined that the results achieved in other organizational units may be useful diagnostic tools to managers in assessing the performance of their own organizational units. Third, with the implementation of the balanced measurement system, the Service does not believe there will exist the same competitive environment where some offices strived for the highest numeric results concerning production measures only. The restrictions established for ROTERs, however, are still in effect. That is, ROTER information concerning one organizational unit may not systematically be shared with other units at the same level. Moreover, those provisions in IRM 1.5, Managing Statistics in Balanced Measurement System Handbook, concerning the sharing of ROTERs within the particular Operating Division and Functional Units are still in effect.
  2. The performance of any one unit at any level of the organization should not be used as a standard by which the performance of any other unit would be evaluated due to differences that exist in the types of taxpayers served, employee skill levels, specific tax issues being worked, and other factors. However, results may be used to identify and share best practices.

1.5.1.11.4 (10-01-2000)
Evaluating the Performance of an Organizational Unit

  1. In conducting a formal review of an organization’s performance, the balanced measures will be the indicators used to determine achievement of the Plan and established goals.
  2. The numeric results achieved with any of the balanced measures can be communicated orally and in writing in an organizational review.
  3. The inclusion of that information provides a point of reference, or starting point, for a more detailed discussion of the actions that were taken to help achieve the IRS’ mission and strategic goals as translated through the balanced measures.
  4. The primary focus of organizational reviews should be on the actions taken and related accomplishments, not numeric results. Reviews should also be forward-looking - using the information and results obtained to identify plan revisions and improvement opportunities.
  5. Diagnostic tools may be used to help explain factors that may have affected the balanced measures results.

1.5.1.11.5 (10-01-2000)
Evaluating Performance of an Individual

  1. Individual managerial or employee evaluations will not be directly tied to balanced measures results.
  2. Non-managerial evaluations will be based upon critical elements or performance standards as appropriate and the review of work performed.
  3. Managerial evaluations will be based on the actions taken in accordance with an agreed upon plan and performance standards. Measures are used in managerial evaluations as an analytical input to determine whether the actions taken achieved the desired performance result. They are not be used as a stand-alone evaluative tool.

    Note:

    This guidance does not apply to the “Pipeline” area of Submission Processing where individual quality/quantity information is used consistent with existing work agreements.

  4. Using this guidance, it would be inappropriate for any written evaluation or performance discussion of an individual to reference a specific number, (i.e. “John Smith met the performance goals established in his managerial performance plan. His office closed 500 more cases than the goal.” ) An appropriate reference might be, (i.e. “John Smith achieved the performance goals established in his managerial performance plan. The agreed upon actions resulted in improvements in customer satisfaction, employee satisfaction, and business results.” ) This overview would then be followed by a more detailed description of the specific actions that were taken toward achieving the goals.

1.5.1.11.6 (10-01-2000)
Reporting Results Externally

  1. While internal management will need to collect enough measures to capture all aspects of performance, external policy makers will be primarily interested in the success of the program. Therefore, for external reporting purposes, measures that focus on results and the desired achievement of a program or service are most useful to decision-makers. In identifying the key or vital few measures to report, an important selection criterion is the integrity of the measures being considered. Because measures reported by the IRS externally are subject to extensive evaluation and review, an especially rigorous set of standards should be met for this set of measures. (Figure 1.3 provides examples of possible measures selection criteria.) In general, it is always in the best interests of any organization to ensure the validity and integrity of all measures to ensure that decision-making is based on accurate, reliable data.Figure 1.5.1-3
    Meaningful - Is the measure significant and directly related to the mission and goal? (Example: If the goal of a program is to provide service to taxpayers, measures that tell how well those taxpayers were served are most meaningful.)
    Valid - Does the measure accurately represent what is being measured? (Example: A measure of customer satisfaction that is based only on internal data does not fully reflect what is purported as being measured.)
    Responsibility Linked - Is the measure matched to an organizational unit responsible for achieving that measure? (Example: a measure of quality aggregated from various programs and services across business units is not responsibility-linked as no clear organizational owner has responsibility for improving the level of performance reflected in the aggregate measure.)
    Customer Focused - Does the measure reflect the point of view of customers and stakeholders? (Example: Standards used to determine the quality of a work product or service should be based in part on the customers’ definition of quality.)
    Credible - Is the measure based on accurate and reliable data? (Example: Cases selected for a quality measure should be chosen at random to ensure that the resulting score accurately reflects the work performed. Otherwise, the results may be biased.)
    Cost effective - Is the measure based upon acceptable data collection and processing costs? (Example: If the costs of collecting statistically valid data at all levels of the organization outweigh the expected benefits of the information obtained, the organization may choose to limit the measure to higher levels or require a lower level of precision.)
    Comparable - Is the measure useful for making comparisons with other data over time? (Example: An organization might consider basing an internal quality score on the same sample of cases/calls used to measure customer satisfaction in order to compare internal results to external perceptions over time.)
    Simple - Is the measure easy to calculate, interpret and understand? (Example: A measure comprised of an index of various sub-measures that requires extensive descriptions and/or footnotes to explain is often difficult for external audiences to understand.)

1.5.1.12 (10-01-2000)
“Getting Behind the Numbers”

  1. Using balanced measures, “getting behind the numbers” refers to any process used to understand changes in performance, determine root causes, and identify improvement options.
  2. There are several methods one can use to “get behind the numbers.” The seven step process outlined below is an example of one process that the IRS has used effectively.
  3. Another process gaining ground at the IRS is the hypothesis-driven problem solving approach. In this approach, one of the first steps is the development of a hypothesis that is used to help formulate tentative conclusions on problems or opportunities and to focus data collection and analysis. Selection of a particular process is dependent upon the situation, level of urgency, time sensitivity, staff skills, etc. Different situations call for different approaches.

1.5.1.12.1 (10-01-2000)
Traditional Problem Solving Model

  1. The following, Traditional Problem Solving Model, describes the seven (7) step process that the IRS uses to “get behind the numbers.”
    1. Receive Data
      The first step is to receive data. Data may be obtained from a variety of sources, examples of which are performance measures results (i.e. survey results, Quality Review Results, data on BPMS (Executive Management Support System), data collected by the Taxpayer Advocate, and Diagnostic Indicators.) In other instances, data may have to be locally developed.

      Note:

      This step is often repeated again after a problem has been defined. For example, an initial scan of data, such as survey results, may suggest that a problem exists with customer service and lead to the development of an initial problem statement which requires additional data collection in order to more accurately define the actual problem.

    2. Define the Problem
      1. In some cases, the problem may be obvious. In other cases, you will need to analyze the data to identify the problem.
      2. If you suspect a problem, then ask the question, “What is the real problem?”
      3. State the problem in objective terms. An accurately worded problem statement is important for the other steps in the process.

        Example:

        “Taxpayers do not have a sufficient understanding of IRS procedures.”
        Not: “IRS employees are not explaining our procedures sufficiently to taxpayers.”

    3. Determine Potential Causes of The Problem
      1. Determining potential causes requires research. Look at all the data available to you. Talk with your employees and peers, and search out potential causes.
      2. Assess the possible causes. Don’t jump to conclusions about the solution.
      3. Evaluate causes and prioritize them based upon their impact on the problem.
    4. Define Courses of Action to Address Identified Causes: Balance Checking Matrix
      1. In this step you will be brainstorming to identify solutions that will address the most significant causes.
      2. Think creatively. Write down all possible solutions. Remember to include “do nothing” as a potential course of action and evaluate the impact of doing nothing in the Balanced Checking Matrix.
      3. Once you have identified possible solutions use the Balance Checking Matrix (See Figure 1.4) to consider their impact on all three balanced measures. (Form 12302 is also available and can still be used.)
      4. Remember, the Balance Checking Matrix is not a “decision matrix” in that you should not simply select the alternative with the most pluses. It is intended to ensure that each Balanced Measure area is considered. It also helps identify any Balanced Measure area where you may need to do something else to reduce possible negative impacts.
      5. When working through the matrix, remember: For each alternative, assign a positive, negative or neutral rating for the impact this alternative might have on each of the three balanced measures:
        A.) Positive impact with a “+”
        B.) Negative impact with a “–”
        C.) Neutral or no impact with “0″
      6. Write out the justification for each rating. This helps to document your reasoning for subsequent evaluation if necessary. For example, if there is a negative impact on employee satisfaction, what is that negative impact?

      Figure 1.5.1-4

      Course of Action: Balance Checking Matrix

      Indicate +, -, or 0 impact on measure and provide rational for rating
      Impact on Measure
      Proposed Courses of Action (State specific cause being addressed) Employee Satisfaction Customer Satisfaction Business Results
      Course of Action 1
      Course of Action 2
      Course of Action 3
    5. Determine a Course of Action
      1. Select a course of action that will have the greatest impact on the problem in order to best support the IRS mission and strategic goals. Communicate the decision in terms of how it impacts employee satisfaction, customer satisfaction, and business results to ensure understanding that all three balanced measures were considered.
      2. In selecting a course of action:
        A.) Consider how the course of action supports the IRS mission and strategic goals
        B.) Consider which course of action may be the most realistic/most cost-effective to implement
        C.) Collaborate with others on making the final selection.
    6. Implement Course of Action
      1. Create a detailed action plan that identifies the steps to be taken in implementing the course of action.
    7. Track Effectiveness of the Course of Action
      1. Your action plan should address how you intend to follow-up and monitor the effectiveness of the selected course of action. This step reinforces the Review and Revise aspects of the Management Model.

1.5.1.13 (10-01-2000)
Proposing, Reviewing and Updating Balanced Measures

  1. Developing good performance measures is an evolving process that improves with time. As such, it is anticipated and expected that there will be changes and/or additions to the balanced measures. When possible, measures changes should occur at the start of the fiscal year. While it is allowable for replacement measures to be implemented during a fiscal year, organizational units should ensure that data continues to be collected for the old measures until the closeout of the current fiscal year.
  2. For measures that are reported externally in the agency’s budget, annual performance plan, or annual program performance report, changes must be coordinated with the national office(s) that are responsible for preparing those documents. Because the measures included in those documents receive substantial scrutiny from oversight bodies, the IRS needs to carefully coordinate and assess the appropriateness of proposed changes to ensure that external requirements or expectations as well as stakeholder concerns are addressed. All measures must conform to Regulation 801 and support the IRS’s Balanced Measures framework.

1.5.1.13.1 (10-01-2000)
Detailed Measures Template Requirement

  1. Prior to proposing any new measure or indicator for inclusion in external and internal documents (e.g., the IRS’ budget or the internal strategy and program plans), organizational units must ensure that the detailed measures template included in Exhibit 1.5.1–6 has been completed.
  2. This template facilitates a common understanding across the organization of a measurement’s definition, formula, reliability, reporting frequency, etc. and is essential in meeting various oversight requirements.

1.5.1.13.2 (10-01-2000)
Examples of Reasons for Changing Measures

  1. The measure isn’t valid (i.e., A measure is not providing data that is truly reflective of the program being measured.)
  2. The measure is no longer meaningful (i.e., A measure was instituted to ensure a problem was corrected. The problem is now corrected and the measure is not providing any new information.)
  3. A better measure exists that will be more useful to management and oversight. (i.e., The existing measure only captured data from two types of casework. The new measure is reflective of all casework completed by the organizational unit.)
  4. The measure is driving the wrong type of behaviors (i.e., The standards used to assess quality of work performed are designed around internal procedural requirements that do not result in quality service to customers.)

1.5.1.13.3 (10-01-2000)
Determining the Need to Update or Modify Measures

  1. The following questions can be helpful in reviewing and updating performance measures.
    1. What adjustments, if any, should be made to the measures currently used?
    2. What developments in the past year will influence current performance measures?
    3. Have there been any problems in measuring performance in the past year?
    4. What changes should be made in the way data are collected and analyzed?
    5. Is the measurement information useful to program management and staff, executive management, the organization’s financial unit, customers and stakeholders?
    6. How could performance reports be enhanced?

1.5.1.14 (10-01-2000)
Integrating Balanced Measures into the Modernized IRS

  1. In order to successfully integrate and use the Balanced Measurement System throughout the IRS, it is recommended that each organizational unit have in place:
    1. A comprehensive approach for measuring and utilizing customer satisfaction, employee satisfaction, and business results (quality and quantity) data;
    2. A clear strategy for using all elements of balanced measures in strategic, operational and business planning; and
    3. A commitment to ensuring, explaining and demonstrating how customer satisfaction, employee satisfaction, and business results are being used and addressed in the business decisions of the organizational unit.

1.5.1.15 (10-01-2000)
How Leaders Can Successfully Use the Balanced Measurement System

  1. “What you do” speaks more loudly than what you say. There are many actions that can be taken to demonstrate commitment to Balanced Measures and to create a work environment that equally fosters Employee Satisfaction, Customer Satisfaction, and Business Results.
  2. The first place to start is to clearly communicate your commitment by using the Balanced Measures approach in all your dealings with customers and employees. When making business decisions, ask “How will this impact customers, employees, and business results?”
  3. Overall:
    1. Be a role model and let others see you using a balanced approach in decision-making.
    2. Communicate decisions in terms of how the decision impacts employee satisfaction, customer satisfaction, and business results.
    3. Mentor others. Help them make balanced business decisions, and assist them in developing plans to improve performance.
    4. Start rewarding, verbally and in writing, those who model the new behavior.
    5. Be an advocate with upper-level management for the issues outside of your control. Communicate to employees what you have done.
    6. Roll up your sleeves and get behind the numbers. Get involved.

1.5.1.16 (10-01-2000)
Additional Information on the Balanced Measurement System

  1. For additional information about the IRS Balanced Measurement System, refer to the Balanced Measures site on the IRWeb at: http://www.hq.irs.gov/Bm/measures.htm

Exhibit 1.5.1-1 (10-01-2000)
General Questions and Answers about the Balanced Measurement System

Q1. How do these organizational measures link to individual front-line appraisals?
Individual appraisals of front-line employees will continue to be based on critical elements for their positions. The critical elements in some areas will or have been adjusted to more closely reflect the Service’s new priorities as reflected in the Balanced Measures. All employee standards have been updated to reflect the retention standard for all employees that requires the “fair and equitable treatment of taxpayers.” Overall, evaluations of individual employees must be based on a review of actual work performed as judged against elements and standards and with consideration given to the specific facts and circumstances of each case.
Q2. How do these organizational measures link to individual manager appraisals?
The balanced organizational measures will be inputs to individual manager performance appraisals but these appraisals will continue to be based on managerial performance evaluation plans. The focus will be on the actions taken to improve performance, not on the numbers.
Q3. What guarantees are there in the process to prevent managers from relying on “numbers?”
Managers will be expected to look not just at the measurement results but at the facts, circumstances, and specific situations in any area that the measures indicate warrant attention. Managerial expectations have been created to support this change and additional steps to reinforce the new approach include changes to the review processes and the individual performance management processes.
Q4. Will we compare our numerical results against prior years to assess progress?
Measurement results from one year will be compared with prior year results to assess progress. This does not mean that a manager’s appraisal is determined by how measures results change over the year. The results can change for numerous reasons, many of which are not under the control of the manager. The actions taken by the manager to improve performance will be what influences the manager’s appraisal.
Q5. How will we measure Customer Satisfaction?
The Customer Satisfaction measure is based on the customer’s perceptions of the service they receive, as obtained through processes such as phone surveys or mail surveys. Survey responses to specific questions will provide the basis for identifying areas with the greatest potential for improvement.
Q6. How will we measure Employee Satisfaction?
The employee satisfaction ratings to be given within the IRS are determined on the basis of information gathered via survey. All employees have an opportunity to provide information regarding employee satisfaction under conditions that guarantee them anonymity.
Q7. How will we measure Quality in Business Results?
Quality will generally be based on an independent review of closed cases or ongoing case work using many of the existing systems ( e.g., EQMS - Examination Quality Measurement System, CQMS - Collection Quality Measurement System, CQRS - Centralized Quality Review System).
Q8. How will we measure Quantity in Business Results?
The quantity area of Business Results will consist of outcome neutral measures that will be used to measure outreach activities and the amount of work completed (e.g., cases closed). Quantity measures may be used as an input in evaluating the performance of an organizational unit as long as the evaluation also considers measures of customer satisfaction, employee satisfaction, and quality.
Q9. How will the IRS know when the measures and goals for customer satisfaction, employee satisfaction, and business results are in balance?
The elements of the Balanced Measurement System each represent an important aspect for assessing progress toward the organization’s goals. Any activity involving balanced measures, such as setting goals, assessing progress, and evaluating results, must consider all three elements. While there is no formula to determine equilibrium among the measures, the impact of the actions taken by the IRS will be reflected in the measurement results and will help shape future plans and strategies for overall performance. In any given year, the mix of improvement programs and strategies proposed should cover all three elements. The purpose of the IRS’s balanced measures approach is to ensure that each element is given due consideration. Working within a framework of limited resources, the senior management team must address some of the most pressing and critical issues by prioritizing and then selecting a mix of strategies and programs aimed at achieving overall progress toward the mission and strategic goals of the IRS.
Q10. How will the IRS use the results of performance measures to set budget priorities?
The IRS will use balanced measurement results to assess the effectiveness of strategies and actions undertaken to improve performance, and those results will serve as one of several inputs into the budgeting process. Measurement results alone, however, are not sufficient for determining future resource allocations. In addition, comparisons among organizational units that focus only on results are often problematic due to differences that exist in the types of taxpayers served, employee skill levels, specific tax issues being worked, and other factors. In some instances, the measurement results may also point to needed changes in internal processes or procedures for which additional funds may not be necessary. Overall, measurement results, detailed analysis of factors affecting program performance, and an assessment of existing and emerging trends, issues and problems will be utilized to determine the appropriate mix of program strategies and allocation of resources necessary to achieve the organization’s goals.
Q11. Does the balanced measurement system limit organizational units from putting in place procedures for prioritizing work or setting standards for certain processes and procedures?
No. The Balanced Measurement System is not intended to limit the IRS from developing work plans, establishing projections for workload indicators, issuing procedures for the manner in which work will be performed, and setting standards on the prioritization and timeliness of work in order to ensure efficient workload management and quality service to customers. The goal of the balanced measurement system is to ensure that business decisions such as these are made in consideration of each element of the balanced measures framework. Therefore, in the establishment of timeliness standards, for example, the business needs to ensure that the organization is appropriately resourced and equipped to meet those standards so that employee satisfaction, customer satisfaction and quality are not negatively impacted.
Q12. Is it okay to discuss failure to meet aspects of a workplan or business plan with a manager?
Yes. However, the conversation must include a discussion of what factors may have contributed to the manager not being able to meet the plan and what actions the manager took and/or should have taken to rectify the situation. There may be a very good reason the plan wasn’t met or there may be new issues that require a revision to the plan, a reallocation of resources, or a change in priorities. Finally, during the discussion, be sure to consider and relate the impact of the performance issue being discussed as well as the impact of potential corrective actions to the balanced measures elements of business results, customer satisfaction, and employee satisfaction
Q13. Can we tell employees that a service standard for the telephone assistance program is that taxpayers should not be left on hold for longer than five minutes?
Yes. In the interests of customer service and business performance, the IRS needs to establish organizational timeliness standards in order to support improved service to all customers. Such standards should be based on the level of resources available and the length of time necessary to provide top quality service to each caller. The agency should not establish standards it cannot meet. Doing so leads to sacrifices in quality and poor service to customers. The focus of any evaluation against timeliness standards should not be on the number but on the appropriateness of the actions that were taken. Obviously, in some instances it will be necessary to leave taxpayers on hold for longer periods of time.
Q14. Is it OK to rely on system data or reports to evaluate front-line employees?
No. System data (e.g., talk-time, call histories, sign-on and sign-off times, etc.) should serve as an diagnostic indicator of where performance issues may exist which the manager needs to then follow-up on and, through his/her own review and monitoring of the employee’s work, determine whether an actual performance problem exists. For example, system data alone could suggest a performance problem, but upon further review and monitoring, a valid or appropriate cause for the system results might be found.
Q15. Why is cycle time or overage considered a quality measure and not a quantity measure?
Under the balanced measurement system, quantity measures provide information about the amount of work performed and products or services provided. Quality measures provide information about the extent to which work performed met prescribed standards, including accuracy, timeliness and completeness. Therefore, cycle time and overage measures assess the timeliness in which work is completed. They do not tell us how much work was completed.

Exhibit 1.5.1-2 (10-01-2000)
Glossary of Terms

Glossary of Terms
Action Plans Tactical plans that identify actions to be accomplished by organizational components to support and implement the Business Plan initiatives and activities. They describe specific actions and objectives being supported as well as the expected performance results.
Balanced Measures Indicator of results for two or more measures considered of equal importance.
Business Plans The Business Plan is developed based on the Strategy and Program Plan and provides a more specific description of major initiatives planned at various organizational levels within each operating unit. The plan also describes specific actions, milestones, responsible parties, performance expectations, and the available resources.
Business Results Measures Indicators of the quality and quantity of work performed.
Customer Any internal or external person or entity to whom you provide services.
Customer Satisfaction Measure An indicator of the level of overall satisfaction with service provided by the IRS as perceived by internal or external customers.
Diagnostic Tools Indicators that are used to discover the factors impacting changes in the Balanced Measures.
Employee Satisfaction Measure An indicator of the level of overall satisfaction with the work environment as perceived by employees.
Goal A desired performance objective, can be qualitative or quantitative (numeric).
IRS Balanced Measures Indicators of organizational performance for Customer Satisfaction, Employee Satisfaction and Business Results.
Organizational Measures Indicators of the progress the IRS is making in achieving its strategic goals and mission.
Outcome Neutral Production or resource data that does not contain information regarding the tax enforcement result reached in any case. (e.g., number of cases closed, level of service provided, assistance and outreach efforts undertaken.)
Performance Measure An indicator of results for an activity, process, organization or program.
Performance Measures Result Numeric outcome of a measure for an activity, process, organization or program.
Quality Measure A numeric indicator of the extent to which completed work meets prescribed standards.
Quantity Measure An indicator of outreach efforts, outcome neutral productivity, and resource utilization.
Stakeholders Groups or individuals who have a vested interest in an organization. They can be internal or external to IRS.
Strategic Goal A strategic goal defines how an organization will carry out its mission over a period of time. It is expressed in a manner that allows a future assessment to be made on whether the goal was or is being achieved. The goal may be of a programmatic, policy, or management nature.
Strategy and Program Plans The Strategy and Program Plan sets the strategies for each Operating Division and functional unit of the IRS. It describes operational priorities and improvement projects for programs being established by the Program Plan. Program plans are an integral part of the Strategy and Program Plan. They contain specific information about the scope and resource allocations that are proposed for the programs commonly defined across the IRS. The Strategy and Program Plan also guides the development of the annual budget submission, where resources needed are specified including total personnel and financial resources for all programs. The final Strategy and Program Plan serves as the guidance for developing the Business Plan.

Exhibit 1.5.1-3 (10-01-2000)
Types of Performance Measures

Types of Performance Measures
Inputs Measure the amount of resources needed to provide particular products or services.
Inputs include labor, materials, equipment and supplies and can also represent demand factors, such as the size of target populations to be served. Input measures are useful in showing the total cost of providing a product or service, the mix of resources used to provide the service, the demand for services, and the amount of resources used for one service in relation to the other services.
Examples Number of customers requesting service, number of FTE or amount of resources applied to a program, amount of supplies available.
Outputs Measure the amount of products or services provided.
Outputs focus on the level of activity in a particular program. Workload measures, which are designed to show how staff time will be allocated to respond to service demand, are most commonly reported. Outputs are useful in defining what a program produces. However, they are limited because they do not indicate whether the program goals have been accomplished, and they do not reveal anything about the quality or efficiency of the service provided.
Examples Number of taxpayers served, number of calls answered, number of cases closed, number of returns processed, number of refunds issued.
Outcomes/Effectiveness Measures that assess how well a service goals and objectives are accomplished. They indicate the quality or effectiveness of a service.
Outcomes reflect the actual results achieved, as well as the impact or benefit, of programs. Policy makers and external oversight groups are generally most interested in outcome measures. Information about the ultimate result is not always readily available or practical to measure. In these instances, it may be necessary to use proxy or surrogate measures.
Examples Field examination quality score, toll free customer satisfaction score, tax law accuracy rate, voluntary compliance, level of service, reduction in incidence of tax fraud.
Efficiency Measures of productivity.
Efficiency measures inform judgments about how well resources were used to achieve intended aims by comparing input indicators with output and outcome indicators.
Examples Calls answered per FTE, average cost per return processed, number of customers served compared to number requesting service.

Note:

Productivity measures should be communicated in a manner which does not encourage organizational units to strive for the highest numerical results at the expense of other Balanced Measures.
Means-Type (Enablers) Qualitative method for describing the means or strategies that an agency will use to achieve its performance goals and indicators
Means type measures typically cover processes, technologies or certain types of resources that will be applied to help achieve a program or operational goal.
Examples Description of how management and oversight organizations support achievement of business unit goals.

Exhibit 1.5.1-4 (10-01-2000)
Examples of Balanced Measures

This image is too large to be displayed in the current screen. Please click the link to view the image.

Exhibit 1.5.1-5 (10-01-2000)
IRS STRATEGIC PLANNING AND BUDGETING CYCLE

This image is too large to be displayed in the current screen. Please click the link to view the image.

Exhibit 1.5.1-6 (10-01-2002)
DETAILED MEASURES TEMPLATE (This template may be updated over time to reflect changing needs. Check with the National Headquarters Office responsible for Servicewide measures and plan development for the latest version.)

DESCRIPTION
Operating Division (s) Operating Division which owns this measure
Measure Name Name of measure/indicator
Type of Measure Identify if this is a Strategic Measure, Balanced Measure, Diagnostic Tool, Workload Indicator, or other non-performance statistic. Identify the component of the Balanced Measurement System that this measure applies to: Customer Satisfaction, Employee Satisfaction, Business Results Quality or Business Results Quantity).
1. Program Category
2. Related Strategic Goal		 1. Identify program category of performance measure (or Budget Activity Level), such as Pre Filing Services, Filing and Account Services, or Compliance Services.
2. Identify the strategic goal that the measure supports, (i.e. Service to Each Taxpayer, Service to All Taxpayers, or Productivity Through a Quality Work Environment).
Responsible Official Example: SB/SE: Director, Strategy, Research & Performance Management
Definition Provide a clear narrative explanation of the measure that can be easily understood by non-IRS reviewers.
a. Reporting Level (s)
b. Report Data Source
c. Reports		 a. Identifies the organizational level(s) for which the measure/indicator is being used (e.g., OD, Area, Center, Territory, Division, Branch, etc..
b. Examples: Integrated Data Retrieval System (IDRS), Command Code ISTSR
c. Examples: BPRS (Business Performance Review Summary), MBPS (Monthly Business Performance Summary) and Strategy and Program Plans
Formula/Methodology Provide a detailed narrative describing exactly how the measure/indicator is calculated, including overall approach, scoring methodology, reporting period, etc. (e.g., rolling averages for period ending….), important status codes, if any, special cases to be considered (e.g., suspense items), etc.
Data Source/
Measurement Tools		 Identify the report or management information system that provides the data and identify the mechanism by which the data being reported is collected (e.g., survey, sample reviews, audit, etc.).
Reliability of Data Describe the process or procedures used to verify and/or validate the data collected. Then based on the following qualifiers, please list the appropriate term: “Reasonable Accuracy” or “Questionable or Unknown Accuracy” to describe data quality of the measure.
Reasonable Accuracy: There is a reliable system or process in place that validates or verifies the accuracy of the data being reported.
Questionable/Unknown Accuracy: A reliable system or process to validate or verify the accuracy of data is lacking. In such cases, a statement should be added regarding efforts underway to improve the reliability of reported data.
Frequency of Data Availability/Reporting Frequency with which measure/indicator data is available and reported, (daily, weekly, monthly, etc.). If the measure/indicator is reported for periods different (i.e. less often) than when the data is available, please explain.
Purpose of Measure: (Describe how measure applies to the segment of IRS operations being measured)

Data Limitations: Describe any limitations for the measure data. (An example of a limitation might be shown as: the data for ACS Taxpayer Delinquent Account Closures does not include the counts for inventory removed systemically.)

Calculation Changes: Describe any approved changes in the formula/methodology made during the fiscal year and the purpose for change (An example this would be a change to the methodology to back out calls to automated systems and better reflect the number of calls being handled by live assistors)

Complete Description of the Process(s) measure originates from:

Critical Path:
1.
2.
3.
4.
(add additional items if necessary)

Management Controls for items on critical path (supervisory/management reviews of data quality, include description of how supervisor/manager indicates completion of a review) If there are no supervisory/ management reviews describe how data reliability, completeness and accuracy is assured:
1.
2.
3.
4.
(add additional items if necessary)
Permalink Print