part7-12
- 7.11.3.1
Introduction - 7.11.3.2
Background - 7.11.3.3
TEQMS - 7.11.3.4
Quality Assurance Staff (QAS) - 7.11.3.5
TEQMS Case Selection Process - 7.11.3.6
Statistical Validity Level/TEQMS Sample Selection - 7.11.3.7
TEQMS Review Process/Quality Standards - 7.11.3.8
Consistency Reviews To Ensure Data Reliability - 7.11.3.9
Case Return Criteria Overview
-
This IRM describes the TAX EXEMPT QUALITY MEASUREMENT SYSTEM (TEQMS) for management and employees in the Tax Exempt and Government Entities (TE/GE) Division. TEQMS replaced the former EP/EO
Quality Measurement System (EQMS). -
TEQMS is designed to measure the quality of the work product demonstrated by TE/GE employees in accordance with established
quality standards for Employee Plans (EP) determination matters. TEQMS will serve as the quality measurement within the business
results component of the Balanced Performance Measurement System. -
TEQMS will allow management to use quarterly/yearly reports to identify improvement opportunities. The officials with the
primary responsibility for analyzing and taking appropriate action on the TEQMS reports are the managers with responsibility
at the corresponding validity level. TEQMS is designed to achieve statistical validity at the national level. -
TEQMS Employee Plans examinations are found at IRM 4.71.15.5, TEQMS Review.
-
Information provided in this section includes:
-
An overview of TEQMS, including changes resulting from the Balanced Performance Measurement System
-
The Quality Assurance Staff/Role of TEQMS Analyst
-
The case selection process
-
The review/upload process
-
Description of reports
-
Tools for assessing data reliability
-
Case return process
-
-
The impetus for developing the new quality standards was based on the Services modernization efforts coupled with the enactment
of the IRS Restructuring and Reform Act of 1998 (RRA 98). The Balanced Performance Measurement System, which includes quality,
is one of the five key modernization “levers of change”
included in RRA 98. The levers are:-
Revamped business practices
-
Four operating divisions
-
Management roles with clear responsibility
-
Balanced measures of performance
-
New technology
-
-
Balanced measures of performance is a three-part concept which is used to assess
organizational
performance including:-
Business results –
quality
and quantity -
Customer Satisfaction
-
Employee Satisfaction
-
-
The effect of the changes to the law enacted by RRA 98 is the Services desire to focus on customer service and satisfaction.
This increased awareness has been incorporated into TEQMS.
-
What TEQMS is:
-
Important component of Balanced Performance Measurement System
-
Determines Organizational measurements
-
Measure of quality as it relates to cases
-
An indicator of improvement opportunities
-
A tool for assessing various, specific components of merit and non-merit cases
-
Evaluative at the national level
-
A method of providing annual and interim quarterly quality reports
-
-
What TEQMS is not:
-
It is not evaluative at the group level
-
It is not evaluative at the employee level
-
-
In general, cases will not be returned and input forms or UFILL record (electronic checksheet) will not be returned.
-
In addition to serving as an organizational quality measure, TEQMS reviews are expected to result in the following benefits:
-
Improve customer service and satisfaction
-
Reduce the frequency of returned cases from Appeals for additional work
-
Help improve the suspension rate in Appeals
-
Increase consistency in determination reviews
-
-
Quality results are reported in a number of high level documents. Examples include the following:
-
IRS Congressional Justification
-
TE/GE Strategy and Program Plan
-
TE/GE Business Performance Review Process
-
IRS Annual Performance Plan
-
Commissioners Monthly Report
-
IRS Management Board Report
-
Executive Management Support System
-
-
Managers and reviewers will support TE/GEs objective to conduct, on a timely basis, quality reviews for each selected determination
application. -
QAS is staffed by a manager and reviewers including a TEQMS analyst. All TEQMS reviews will be performed by QAS personnel.
-
Reviewers are specialists who have demonstrated a comprehensive knowledge of accounting and auditing principles, tax laws,
IRS policies, and internal procedures. Reviewers must be objective and exercise independent judgement and individual initiative
in the performance of their duties. Reviewers are charged with reviewing determination cases and reporting the results of
those reviews via completion of a UFILL disk (electronic checksheet) that contains the appropriate input form. Reviewers are
expected to effectively communicate with other technical personnel, management, taxpayers and external customers when providing
or obtaining information to timely resolve issues in completing their assignments. -
The role of the TEQMS analyst includes oversight of the sample selection process, monitoring receipt of sample cases, preparation
of quarterly/yearly reports and implementation of consistency reviews and validity reviews, as appropriate. For EP determinations,
the TEQMS analyst serves a dual role as both a senior reviewer with normal Quality Assurance duties and as the TEQMS analyst.
-
EP determination cases subject to TEQMS review are broken down into the following two classifications:
-
EP Determinations – the population consists of all cases, including mandatory review cases, except: volume submitter specimen
plans, master and prototype specimen plans, pooled trusts, non-lead multiple employer plans, and technically screened cases.
Each unique combination of “EIN and plan number”
is treated as a separate unit in the population. Cases that are not worked, such as returned incomplete, withdrawn by
taxpayer, substantially deficient, cases re-selected for TEQMS review, and correction disposal are not included in the population.
-
EP Determinations/Technically Screened Cases – this population consists of all cases closed on merit with or without contact,
except for non-lead multiple employer plans, pooled trusts, and other designated plans to be determined. Each unique combination
of “EIN and plan number”
is treated as a separate unit in the population.
-
-
At the beginning of each fiscal year, the size of each population described above is determined based on the number of cases
expected to be closed during the year. An annual sample size is calculated for each population using statistical calculations
(95% level of confidence and margin of error of plus/minus 5 percent). Results from determination case reviews are statistically
valid at the national level; the actual precision of the quality data is computed at year-end. -
Cases are sample selected automatically by the EP/EO Determination System (EDS) during the group closing process. The sample
selection rate is monitored during the year and adjustments made as deemed appropriate. If the automated sampling process
does not function properly, a manual process will be instituted with the assistance of the Data Analysis Unit (DAU) in ensure
the statistical validity of the sample
-
Cases are reviewed in open status and then closed by QAS after TEQMS review.
-
Reviewers will review cases and complete an input form or UFILL record (electronic checksheet) that contains the appropriate
input form. The input form is composed of two sections, an administrative portion and a quality standards/elements portion.-
The administrative portion contains fields to capture review information, case return criterion and case information.
-
The input forms for EP Determinations contain quality standards, elements, aspects, and reason codes. The input forms for
EP Technical Screening contains only quality elements and reason codes.
-
-
Specific instructions for completing the input form and guidelines for applying the quality standards, elements, aspects,
and reason codes for EP determinations are found in the TEQMS – Tax Exempt Quality Measurement System – EP Determinations
(Student Guide), catalog number 86711Q.
-
EP determination UFILL records (electronic checksheets) will be uploaded to the TEQMS database by QAS personnel.
-
The database performs a series of validity checks and a report is generated on the status of each record. QAS personnel are
responsible for resolving all errors.
-
The data uploaded to the TEQMS database for EP Determinations and EP Technical Screening will be summarized in the form of
various reports that are generated each quarter. These reports provide an indicator of the extent to which completed work
meets quality standards. The reports:-
Provide measures of overall performance on standards.
-
Identify potential improvement areas.
-
Provide feedback on specific root causes.
-
Identify potential training/CPE needs.
-
Do not provide evaluative information for individuals.
-
-
The reports include the following:
-
Percentage Yes (answers) by Standard/Average Total Score – reflects the percentage of yes answers for each standard as well
as the organizational score. -
Percentage Yes (answers) By Standard/Average Total Score By Quarter– reflects the percentage of yes answers for each standard
as well as the organizational score for each quarter and cumulative fiscal year. -
National Quality Standards – reflects the percentage of yes answers for each standard/element/aspect.
-
National Quality Standards Detail – reflects the number of yes, no, and n/a answers for each standard/element/aspect as well
as reason codes/process measures. -
Case Return Criteria – reflects the number of cases returned pursuant to the return criteria.
-
Narrative comments for yes and no answers.
-
-
The database-generated reports will be analyzed and trend reports prepared. The reports will contain a discussion of where
the EP determinations organization has or has not done well, suggestions for improvement and a trend analysis based upon the
answers to yes and no questions on the input form. A separate report is prepared for EP Determinations and EP Technical Screening
cases.
-
The Balanced Performance Measurement System requires that organizational case quality be numerically scored. This measure
will serve as an indicator of the progress the Service is making in achieving high-level strategic goals. TEQMS provides a
rating system that will best measure case quality for the organization. -
TEQMS should be distinguished from an individual measure. Manager case reviews should assess individual performance strengths
and weaknesses based upon negotiated critical elements. The TEQMS standards are closely associated with critical elements. -
The TEQMS database computes the organizational score based on the reviewers answers concerning whether the case met each
standard (yes or no). For EP Determinations there are 6 quality standards. Some standards contain elements and/or aspects.
The maximum total score for the organizational equals 100. For EP Technical Screening there are 5 quality elements, which
may contain reason codes, and the maximum total score for the organizational equals 100. (For a detailed discussion of the
quality standards, elements and aspects for EP Determinations and Technical Screening, refer to the TEQMS – Tax Exempt Quality
Measurement System – EP Determinations (Student Guide), catalog number 86711Q).The organizational score for
all
EP Determination cases is calculated as:Total Number of Standards Passed / Total Number of Standards Rated X 100 = Organizational Score
The organizational score for
all EP Technical Screening cases is calculated as:Total Number of Elements Passed / Total Number of Elements Rated X 100 = Organizational Score
Example 1:
During the fiscal year, 400 EP Determination cases are sampled for TEQMS. The EP Determination quality input form contains
six quality standards. Therefore the total number of standards rated equals 2,400 (400 x 6). Of the 2,400 standards rated,
only 1,200 standards are passed (i.e. answered “Yes”
).The organizational score is calculated as:
1,200 / 2,400 X 100 = 50
Example 2:During the fiscal year, 400 EP Technical Screening cases are sampled for TEQMS. The EP Technical Screening quality input form
contains five quality elements. Therefore the total number of elements rated equals 2,000 (400 x 5.) Of the 2,000 elements
rated, 1,800 elements are passed (i.e. answered “Yes”
).The organizational score is calculated as:
1,800 / 2,000 X 100 = 90
-
To encourage greater consistency between reviewers in applying TEQMS standards to cases, the Quality Assurance Manager will
be responsible for implementing consistency reviews on a quarterly basis. The TEQMS analyst for EP determinations is responsible
for scheduling and administering these consistency reviews. On a quarterly basis, the analyst will select at least one EP
Determination case and at least one EP Technical Screening case to be rated independently by each reviewer. The analyst will
also review the case(s). Each reviewer and the analyst will prepare the input forms. After each reviewer has reviewed the
cases, a copy of the input form will be provided to the analyst. The analyst will tabulate the results from each input form
and prepare a summary workpaper of the results to be distributed to each reviewer. -
The analyst will also schedule a meeting to discuss the conclusions reached by each reviewer with emphasis on the standards,
elements and key aspects for which there were differences. It is desired that a consensus will be reached on these differences
by all reviewers. If this is not possible, this will be noted in the report with a recommendation for appropriate action,
such as additional training or clarification of a procedure. The analyst may prepare a report on the consistency review and
forward this report to the Quality Assurance Manager for approval. After approval, this report will be filed by the TEQMS
analyst and disseminated as deemed appropriate. The TEQMS analyst will also be the primary reviewer on the cases selected
for this review and be responsible for closing the case, including the issuance of a reviewers memorandum and/or correction
of the determination letter, if required. The Quality Assurance Manager is responsible for taking appropriate action if the
results of the consistency review indicate that additional training or improved procedures are required.
-
This section provides –
-
Criteria for returning cases to the specialist for further development.
-
Instructions to the TEQMS reviewers for returning cases to specialists.
-
Type of feedback that is prohibited from TEQMS.
-
Type of feedback that is permitted by TEQMS.
-
-
Cases will be returned to the group when any of the following conditions exist. See IRM 7.11.3.9.2 and See IRM 7.11.3.9.3.
-
Mandatory review cases are part of the sample universe considered by this system. They will be returned for correction following
current procedures. See IRM 7.11.1.40.1 for a list of the Mandatory Review cases for EP determinations.
-
A case will be returned to the EP Specialist if there is clear evidence that an incorrect determination has been reached with
regard to the qualification of a retirement plan. An underdeveloped case will not be returned unless, in the case file, it
is evident that the plan is not qualified. -
A case will be returned if there is evidence of fraud, malfeasance, collusion, concealment or misrepresentation by the taxpayer
or representative. -
A case will also be returned if there are other circumstances that indicate that a failure to return the case would be a serious
administrative omission. For example:-
Instances which may result in serious criticism of the Services administration of the tax laws;
-
Establishing a precedent that would seriously hamper subsequent attempts by the Service to take corrective action; or
-
Actions that result in inconsistent treatment of similarly situated taxpayers.
-
-
Mandatory review cases will be returned for correction following current procedures.
-
Cases will be returned if plan qualification not apparent.
-
Cases will also be returned if other circumstances exist that indicate failure to return the case would be a serious administrative
omission. For example:-
Treatment would result in serious criticism of the Services administration of the tax laws;
-
Establishing a precedent that would seriously hamper subsequent attempts by the Service to take corrective action; or
-
Actions would result in inconsistent treatment of similarly situated taxpayers.
-
-
EP Quality Assurance reviewers may correct administrative type errors on EDS closing documents. Quality Assurance reviewers
may also correct errors on closing letters (e.g., improper caveats, EINs, taxable year, incorrect name or address, plan number,
POA letters, advisory paragraphs, etc.). -
Generally, technical corrections will not be made by reviewers. The case will be sent back to the specialist.
-
The Quality Assurance Manager must approve cases that meet the case return criteria, which are returned on a Form 5456,Reviewers Memorandum – TEGE, Employee Plans Determinations.
-
Except as provided below See IRM 7.11.3.9.6, the following feedback may not be generated on TEQMS because this is an “organizational”
measure,
not
an individual performance review.-
Input forms will
not
be returned to the group manager or specialist. -
Commendatory memos will
not
be prepared on TEQMS cases. -
Advisory memos will
not
be prepared on TEQMS cases. -
For cases that meet the case return criteria, the Reviewers memorandum will
not
state the case score.
-
-
When a minor technical or procedural error is found by a reviewer, and the reviewer is able to correct the error, the case
will continue to be processed. The case will not be returned to the Specialist, but effective January 3, 2005 an email is
to be sent to the specialist and the TEQMS analyst regarding each technical screening case that is reviewed for TEQMS, regardless
of whether an error was discovered by the reviewer. The feedback emails may comment on determination letter quality and general
and/or specific comments, based on each individual case review. The email may also contain commendatory or advisory comments. -
Only the specialist and analyst will receive specific case information, this will render the feedback ineligible for managerial
evaluation. The analyst will collect the emails and, on an as needed bases, prepare summary reports for each group detailing
issues or comments for which a trend(s) appears to develop for their group. The management summaries are not specific to any
individual and therefore cannot be used for evaluative purposes. The summaries may be discussed in a group setting. -
Cases containing significant errors will be processed according to See IRM 7.11.3.9.2,
See IRM 7.11.3.9.3, and See IRM 7.11.3.9.4.
