part13-29
- 13.5.1.1
Introduction - 13.5.1.2
The Balanced Measurement System - 13.5.1.3
Using Balanced Measures - 13.5.1.4
Business Performance Review System - 13.5.1.5
Taxpayer Advocate Service Balanced Measures
-
As part of the IRS and its new organization, TAS has incorporated balanced measures in its procedures.
-
In 1998, IRS developed a plan for Modernization that consisted of five
“levers of change”
to modernize the IRS and support the new mission. One
“lever”
involves implementing a system of balanced measures to assist in measuring and improving organization performance. -
The IRS developed its balanced measurement system in an effort to modernize the IRS and to reflect the IRS priorities as
articulated in the IRS mission statement. This system does the following:-
supports the organizational mission and goals;
-
communicates priorities;
-
meets external reporting requirements;
-
provides a balanced approach to measuring performance at each level of the organization; and
-
expands performance measurement to include more than operational financial results.
-
-
To ensure balance, this system measures performance in three areas:
-
customer satisfaction;
-
employee satisfaction; and
-
business results.
-
-
Each of these measurement areas are carefully considered when setting organizational objectives, establishing goals, assessing
progress and results, and evaluating individual performance. -
TAS uses balanced measures to assess program effectiveness and service delivery.
-
The goal of customer satisfaction, one part of the balanced measurement system, is to continually improve the IRS ability
to provide accurate and professional service to internal and external customers in a courteous and timely manner. -
IRS determines customer satisfaction goals and accomplishments on the basis of customer feedback collected via questionnaires,
surveys, and other types of information gathering mechanisms. -
The balanced measures system will help answer key questions regarding customer satisfaction including:
-
the general level of customer satisfaction in specific IRS transactions;
-
the frequency and reason for customer dissatisfaction; and
-
taxpayers perceptions and expectations of the IRS.
-
-
The goal of employee satisfaction, the second part of the balanced measurement system, is to provide a work environment that
enables employees to achieve organizational goals. -
The
“quality work environment”
goal assesses measures of employee satisfaction. -
The balanced measurement system will help answer key questions regarding IRS employee satisfaction including:
-
the general level of employee satisfaction;
-
employees perceptions of the effectiveness of various levels of management; and
-
the employees perception of the IRS provision of an appropriate work environment.
-
-
The goal of business results is to generate a productive quantity of work in a quality manner and to provide meaningful outreach
to customers. This includes delivering a volume of work within mixed categories and dedicating appropriate resources to provide
taxpayer education, outreach, and advocacy. -
The business results measures consist of numerical results determined under the
“quantity”
and
“quality”
elements. -
The goal involves more than achieving a target or predetermined quality and quantity results. It addresses key questions regarding
business results including:-
Did we assess/collect the proper tax?
-
Did we provide quality customer service?
-
Did we use our resources in accordance with the plan?
-
Did we provide education and outreach/advocacy to our customers?
-
-
Each balanced measure element represents an important aspect of the organizations goals and is of equal importance in carrying
out the IRSs program and functions. -
The frequency of measures data availability may vary across the three elements because the data from each element changes
at different intervals and/or requires more time for data collection. However, differences in the frequency of data availability
do not reflect differences in priority of the balanced measurement elements.
-
The balanced measurement system changed the way the IRS uses measures. In the past IRS used measures in ways that produced
unintended consequences, such as ranking business units or functions on an index of measures. -
The emphasis of the new system is on understanding why measured data have changed and what actions IRS took or should have
taken to influence results. -
We are not managing programs to achieve numbers. We are managing processes and people to achieve the IRS mission.
-
The use of quality tools such as flow charts, pareto charts, root cause analysis, etc., prove valuable in the fact finding
process. -
You can find information about the appropriate use of measures, including the definition of record of tax enforcement results
(ROTERs), goal setting use of measures in evaluations, sharing data, etc., in IRM 105.4, Managing Statistics in a Balanced Measurement System. -
All managers must ensure strict adherence to this guidance.
-
Businesses often refer to the process of analyzing business results or understanding the changes in measures as
“getting behind the numbers.”
There is no prescribed method for this although the following guides may be of assistance:-
The Presidents Quality Award (PQA) Criteria for Excellence is an excellent way of evaluating organization success and developing
opportunities for improvement based on comparison with
“best in class”
activity. -
The Management Analysis Resource Guide, based on the Management Model, used in conjunction with diagnostic tools, helps managers understand activities they should
be doing on a regular basis in order to
“get behind the numbers.”
-
-
Typically, we initiate the analytical process by looking at:
-
the balanced measures data and the diagnostic tools data;
-
those more closely involved in the process; and
-
the actions that impacted the results.
Example:
A change in the TAS case cycle time measure may be due to different methods of case progression or to equipment problems.
-
-
It may not be easy to discern the root cause of the change by looking at data alone. An analysis of the facts and a brief
conversation with individuals close to the process may quickly bring this to light. -
You may find detailed information on the balanced measures system in IRM 105.4, Managing Statistics in a Balanced Measurement System, Chapter 2.
-
The IRS collects a great deal of additional information about programs and services, some of which have been used as performance
measures in the past. -
Under the balanced measurement system, a business unit or organization may only use the approved set of balanced measures
— both strategic and operational — to measure its performance. -
IRS refers to indicators that are not designated as balanced measures as
“diagnostic tools.”
Use diagnostic tools to analyze the factors that affect changes in the balanced measures performance and to
“get behind the numbers.” -
Using diagnostics tools provides you with a mechanism to analyze factors that influence performance and encourages dialogue
about specific actions that managers may take to improve customer satisfaction, employee satisfaction, and business results. -
Do not set goals or targets for diagnostic tools and do not use them in individual performance evaluations.
-
Diagnostic tools include any type of data that is helpful in understanding what influences and impacts balanced measures.
It is permissible to use ROTERs as diagnostic tools.
-
The Reference Guide for Diagnostic Tools serves as a managerial resource to help you change the way you review statistics and manage data. Although originally written
for previous IRS functions, the overall framework for diagnostic tools translates to the new IRS organization. -
The guide will:
-
Encourage dialogue by providing a mechanism to help managers analyze the data elements/diagnostic tools that are
“behind the numbers”
and to focus on management actions based on that context; -
Aid management by helping diagnose outcomes to get
“behind the numbers”
and to analyze indicators/data elements and actions taken to address the problems; -
Assist management of organizations or business units to focus on and understand the underlying factors that affect the key
set of balanced measures (e.g., to help separate controllable and external factors); -
Provide information to allow revision of future plans; and
-
Define topic parameters for operations/business review by setting expectations, eliminating surprises, and ensuring the ability
to prepare.
-
-
Some TAS examples of diagnostic tools are listed in Exhibit 13.5.1-1, TAS Diagnostic Tools.
-
Use diagnostic tools to understand underlying factors that cause changes in balanced measures.
-
Do not use diagnostic tools as organization performance measures.
-
Do not set goals or targets for diagnostic tools.
-
Do not use these tools as an individual performance measure (i.e., in evaluating an individual employees performance).
-
Do not use diagnostic tools as a
“gotcha”
in performance reviews. -
Do not compare the results of diagnostic tools when evaluating different offices or units.
-
The Business Performance Review System (BPRS) represents a fundamental change in the way IRS reviews and manages its operations.
-
The system enables a periodic review of strategic and operations issues and business unit performance and facilitates the
assessment of the IRS progress in achieving its mission and strategic goals. -
The BPRS establishes a framework for measuring, reporting, and reviewing a business units performance against plans established
within the Strategic Planning and Budget process. -
During this process each business unit identifies, defines, and tracks the essential elements of its performance.
-
IRS uses BPRS meetings to assess all business units progress toward organizational goals and to identify crosscutting issues
in a timely manner. -
The BPRS provides the opportunity to identify areas of the plan where revisions are necessary, which then can be incorporated
into the next strategic planning and budget cycle.
-
The National Taxpayer Advocate (NTA) approved ten measures. Text below describes in detail all the measures.
-
The measures are broken down into the categories mentioned above — customer satisfaction, employee satisfaction, and business
results (quantity and quality). -
The measures within each category relate to either casework or advocacy. See Exhibit 13.5.1-2, TAS Balanced Measures, for a chart of TAS ten measures and this relationship.
-
The casework quality index is one of the quality business results of TAS balanced measures.
-
This is a numerical indicator of the extent to which TAS casework meets the standards prescribed.
-
These results are indicators of quality and are used to identify and correct problems that require changes in areas such as
procedures and training as well as to identify best practices or systemic problems. -
The results of these reviews are for use by the National Office as well as Area Taxpayer Advocates (ATAs), Local Taxpayer
Advocates (LTAs), and quality analysts only. Do not share specific case review results (i.e., quality review checksheets and comments sheets) with TAS group managers or
Associate Advocates (AAs)/Senior Associate Advocates (SAAs). See Exhibit 13.5.1-3, Sharing Quality Review Results, for more information about sharing quality review statistics and case results. -
The information in the following text describes TAS responsibilities for this measure, the procedures for calculating the
index, and other features of this balanced measure.
-
The Director, Program Planning and Review, in the NTAs office, the ATA, and LTA are responsible for the quality review program
in their respective offices. -
The information in the sections below describes the responsibilities in detail.
-
The Director, Program Planning and Review at the National Office is responsible for the centralized quality review program
for TAS. -
That person is responsible for:
-
ensuring that the review sites review and document the results of the monthly quality samples sent from the field TAS offices
and transmit the quality review database records to the appropriate field sites as well as National Office; -
maintaining and revising, as appropriate, the TAS quality review database (QRDB);
-
generating the national monthly and cumulative quality reports and the data sent to the Executive Management Support System
(EMSS); -
responding to DIALOGUE situations (see IRM 13.5.1.5.1.5, DIALOGUE Process) which the field or review sites elevate to the National Office and disseminating the results to the area TAS offices and
review sites; -
publishing a quarterly quality newsletter through the appropriate TAS communication channels; and
-
providing an annual report on quality consisting of information from area offices.
-
-
Each ATA is responsible for the centralized quality review program for TAS in his/her area.
-
The ATA is responsible for:
-
ensuring that the monthly quality samples are sent from the areas field TAS offices to the appropriate review site;
-
maintaining the areas TAS quality review database (QRDB);
-
generating the areas monthly and cumulative quality reports;
-
being the first level of review for DIALOGUE situations from the field offices in that area (see IRM 13.5.1.5.1.5, DIALOGUE Process, elevating to the National Office any DIALOGUE situations for which there is a disagreement;
-
disseminating the results of nationally elevated DIALOGUE results to the field TAS offices in that area;
-
analyzing quality review data for the area and offices within that area to identify trends, procedures needing improvement,
training needs, systemic problems, and best practices; -
Using the analytical results to improve quality in the areas local offices/campuses (e.g., share best practices, set up area
training classes, work with offices on specific problem areas, etc.); -
sharing systemic analysis results with the Operating Division Taxpayer Advocate (ODTA) for the appropriate business unit;
-
providing input to the quarterly quality newsletter published by the National Office; and
-
providing input (i.e., quality analysis) to the National Office for an annual report on quality.
-
-
The LTA at each local office/campus is responsible for the centralized quality review program for TAS in his/her office.
-
The LTA is responsible for:
-
ensuring that the monthly quality samples are sent from the TAS offices to the appropriate review site;
-
maintaining the offices TAS quality review database (QRDB);
-
generating the offices monthly and cumulative quality reports;
-
submitting DIALOGUE situations for questionable situations to the appropriate review site, keeping the area quality analyst
informed of the situations, and elevating any discrepancies to the National Office (see IRM 13.5.1.5.1.5, DIALOGUE Process) -
disseminating clarifications in TAS procedures to the field TAS managers and caseworkers in the office;
-
analyzing quality review data for the office to identify trends, procedures needing improvement, training needs, systemic
problems, and best practices; -
Using the analytical results to improve quality in the local office/campus (e.g., share best practices, set up training classes,
work with managers and AAs/SAAs on specific problem areas, etc.); -
ensuring cases that should be reopened are correctly resolved;
-
sharing systemic analysis results with the area and ODTA for the appropriate business unit;
-
providing input to the area for the quarterly quality newsletter published by the National Office; and
-
providing input (i.e., quality analysis) to the area for the national annual report on quality.
-
-
At the end of each month the local offices and campuses must select a random sample of their regular and reopen criteria cases.
See Exhibit 13.5.1-4, Quality Sample Sizes for Centralized Quality Review, for sample sizes by office. -
Until TAMIS has a new quality sampling report, use a Vision Query (a/k/a Intelligent Query (VQ)) procedure to list the cases
eligible for sampling. -
See Exhibit 13.5.1-5, Selecting Cases for the Centralized Quality Review Sample, for procedures on selecting the sample.
-
Send the samples by the scheduled due date to the Oakland review site for local offices or to the Brookhaven review site for
campuses. The National Office publishes a quality review schedule one month prior to the beginning of each fiscal year. -
See Exhibit 13.5.1-6, Documentation Required for TAS Cases, for documentation that must be in each case file.
-
The addresses for the review sites are:
-
Oakland — Internal Revenue Service, Quality Assurance Division/TAS Review, Attn: Anita Kitson, 1301 Clay Street, Suite 800S,
Oakland, CA 94612. -
Brookhaven — Internal Revenue Service, PAS — Stop 110, 1040 Waverly Avenue, Holtsville, NY 11742.
-
-
TAS derives its quality index from the monthly quality reviews of local offices and campuses monthly randomly sampled cases.
-
The reviewers at the two sites review the TAS cases following the guidelines in the TAS Quality Standards and Review Guide.
Basically, the quality standards for the quality index reflect the TAS casework procedures required in this IRM. -
There are eight quality standards. The casework quality index is the result of the review of these standards. Since this business
measure is for overall TAS casework, reviewers look at the case in its entirety and do not differentiate between offices when
the case involves transfers from another office. -
Each standard weighs a particular amount of points (shown in parentheses below). The maximum total applicable points for a
case are 100. Not all standards are applicable in every case. You may find a more in-depth description of the standards and
their applicability in Exhibit 13.5.1-7, TAS Casework Quality Index (CQI) Standards. -
The standards are:
-
Did TAS make timely initial contact with the taxpayer? (5 points)
-
Did TAS take initial action/request information within the specified time frames? (10 points)
-
Did TAS take all subsequent actions timely from the time action could have been taken? (10 points)
-
Did TAS resolve all taxpayer issues? (25 points) Did TAS address all related issues? (10 points)
-
Were all adjustments that impact the taxpayer technically/procedurally correct? (15 points)
-
Did TAS give the taxpayer a clear, complete, correct explanation at closing? (20 points)
-
Did TAS educate the taxpayer regarding any of his/her actions that contributed to the problem? (5 points)
-
-
The TAS Quality Review Database (QRDB) is a personal computer based database created using Microsoft Access software.
-
The Taxpayer Advocate office in each local office/campus, area office, operating division, and National Office has a copy
of the QRDB that contains the review results (e.g., checksheet, comments, and closure/sample size information) for the particular
office.Example:
A local offices database such as Greensboros has only the results of a review of Greensboros monthly random samples. The
QRDB for Area 2 — Richmond, of which Greensboro office is a part, contains the review results for all the local offices within
Area 2. The National Offices and the ODTA offices QRDBs contain results from all the offices within the nation. -
The National Office transmits an empty QRDB to the area offices and ODTA office who in turn send it to the local offices/campuses.
A designated administrator in each office (usually the quality analyst) is responsible for loading the monthly results received
from its review site. -
You, as an analyst, ATA, LTA, or ODTA may use the QRDB to generate monthly and fiscal year cumulative reports as well as specific
queries for data analysis.
-
The TAS DIALOGUE is a period of time following a monthly review when local offices and campuses have an opportunity to discuss
errors charged on the previous months reviews. -
This time allows an exchange of information to clarify errors, identify best practices, identify training needs for reviewers
or TAS employees, identify systemic problems, and ensure procedures are being followed correctly. -
Results of the discussion have no impact on the local office or campuss score.
-
You may elevate any situations that cannot be solved at the review site or area level to National Office. Once decided, National
Office disseminates the situation, the TAS offices view, the review sites view, and the national decision to the area and
local offices to ensure all TAS employees are following the same case processing guidelines. -
See Exhibit 13.5.1-8, DIALOGUE Process, for the steps of the process.
-
The text below describes TAS closed cases, one of TAS business results (quantity) measures.
-
The volume of regular criteria closures (i.e., first time TAS contacts meeting one of TAS criteria 1-7) represents one of
TAS business results (quantity) measures. Its purpose is to measure TAS productivity and effectiveness in identifying, working,
and resolving all taxpayer issues and hardships that meet TAS case processing criteria expanded by the enactment of the Revenue
Restructuring Act of 1998. -
In conjunction with the other business results (e.g., customer satisfaction, CQI, and outreach), TAS closed cases will afford
TAS the opportunity to gauge effectively its performance in ensuring that all taxpayers whose issues meet TAS criteria receive
the appropriate TAS assistance, intervention, and case resolution.
-
Until TAS redesigns its TAMIS reports to capture and track TAS new performance measures, the Director, Program Planning and
Quality, has the responsibility to record TAS closed case performance for all offices and to convey performance results for
the Commissioners monthly report and to EMSS. -
Vision Query (a/k/a Intelligent Query) is the vehicle to capture closed case performance results.
-
TAS will share with each area office the national composite results, specific area composite performance, and performance
results for those offices within the specific area. -
Each area office will convey to its local offices the national composite performance, the areas composite performance, and
the individual performance specific to the local offices/campuses under the areas jurisdiction. -
The implementation of the TAMIS reports re-design will provide for the systemic capture and recordation of closed case performance
results and the electronic transmission to EMSS. -
Each Advocate office will also have TAMIS access to the closed case performance results to which it is entitled to view within
the parameters and guidelines of the IRS policy of managing statistics in a balanced measures environment (See IRM 105.4, Managing Statistics in a Balanced Measurement System.)
-
IRS restricts access to closed case statistical data, as well as access to other statistical performance results, in accordance
to its policy of managing statistics in a balanced measures environment (see IRM 105.4, Managing Statistics in a Balanced Measurement System). -
No Advocate office may access and view closed statistical data of a peer office (i.e.,
“horizontal”
access restrictions). An Advocate office may access its own closed case data and composite closed case performance results
of those offices directly above or below it (i.e.,
“vertical”
access permissions). -
The National Office and the ODTA offices have access to all composite and individual closed case performance data.
-
An area office may access national composite data, its own composite data, and individual date for the local offices within
its jurisdiction. -
A local office/campus may access national composite data, composite data for the area to which it belongs, and to its own
individual data. -
A group within a local office/campus may view its own group data, the composite data for the local office, area office, and
nation. -
Plans include reprogramming both TAMIS and Vision Query to ensure systemic adherence to the above mentioned access permissions
and restrictions.
-
The text below describes TAS closed case cycle time, one of TAS business results (quality) measures.
-
Closed case cycle time is one of the quality components of TAS business results measures.
-
TAS uses it to measure timeliness in resolving effectively all taxpayer issues and concerns that meet TAS case processing
criteria. -
Previously, TAS defined this measure as the total days to resolve/close regular criteria cases divided by the count of regular
criteria closures. -
The definition in (3), above, did not capture fully the total time expended to resolve taxpayer issues because it was silent
as to reopen closed case cycle time. A reopened case is an extension of a period case that did not address all taxpayer issues
or did not respond to the issues in a manner deemed satisfactory by the taxpayer. -
TAS has re-defined the closed case cycle time measure as the total days expended to close both regular and reopen criteria
cases divided by the count of regular criteria closures. -
By adding reopened case cycle time to that of the original closed case cycle time, this new cycle time measure accounts more
effectively for and tracks TAS timeliness in addressing and resolving taxpayer issues and concerns.
-
The reporting and recording procedures for closed case cycle time performance results are the same as for TAS closed cases
indicated above in IRM 13.5.1.2.2, Employee Satisfaction Measures.
-
TAS conducts an annual sample survey of externaI customers (taxpayers and their powers of attorney) who have had recently
closed cases. -
The survey gauges customers reactions to key service variables such as:
-
the quality and timeliness of service;
-
the responsiveness, courtesy and professionalism of the TAS employees with whom they dealt;
-
the ease of communications; and
-
the end results (perception of fairness of resolution).
-
-
TAS collects customer satisfaction data as part of the balanced performance system of organizational evaluation.
-
The collected data are for the sole purpose of enabling TAS to assess its customers perspectives of TAS delivery of services.
-
The customer satisfaction survey is part of a reiterative process to drive both local and national planning in each successive
fiscal year to better meet customer expectations.
-
The Director, Program Planning and Quality in the National Office owns the process of managing the customer satisfaction survey.
-
The area offices and their respective subordinate offices own the data and reports produced by the survey for their offices
as well as the nation as a whole. -
ATAs and LTAs are collectively responsible for analyzing the data using process management techniques and for engaging employees
through their representative organizations in identifying local initiatives to improve customer satisfaction. -
Use analysis of customer satisfaction survey data integrated with the other balanced measures to plan and recommend specific
actions (i.e., changes to procedures, training, equipment, etc.) that you need to take to improve satisfaction scores in subsequent
surveys. -
Incorporate improvement objectives flowing from survey analysis into the performance expectations of local and area managers
and consider this as an integral part of the strategic planning and budgeting process for each office.