Anchor: #i1007108

Section 7: Project Evaluation

Anchor: #i1007113

Introduction

Project evaluation is an ongoing process conducted throughout the grant agreement period. This evaluation process is performed at different levels of administration. The following table shows the phases of project evaluation and who performs them.

Anchor: #i1005353Performance of Project Evaluation

Type of Evaluation

Performed by

Periodic project monitoring (described later in this section)

project and program managers

Review of Performance Reports

project and program managers

eGrants project grading

project and program managers

Final project evaluation

project and program managers

Annual Report (covered in Section 8 of this chapter)

TRF-TS planner



Anchor: #i1007122

Periodic Project Monitoring

Data gathered through the project monitoring process (covered in Section 3 of this Chapter) is an important part of evaluation of the project.

Project Performance Reports and Requests for Reimbursement (RFRs) normally give an evaluator some sense of whether or not the project is proceeding according to the project’s objectives in terms of activities reported and expenditures included. Anything that appears out of the ordinary can then be followed up with an on-site monitoring visit. (In addition, on-site monitoring visits provide an opportunity to determine progress and detect problems.) In this way, the evaluator might discover a minor problem early and recommend an early correction in activity or tighter fiscal control to avoid a major problem later.

In addition, the reports should also indicate whether substantial overruns or underruns are occurring, and if so, they should be explained in the report. In the event of substantial underruns, the program manager might recommend reallocation of these resources to other areas where additional funds can be better utilized. For overruns, the program manager should ensure that all costs are expended according to the approved budget and that the grant can sustain itself throughout the effective term period. Otherwise, an amendment to the grant may be required. Project and program managers should submit these recommendations to the Texas Department of Transportation (TxDOT) Traffic Operations Division - Traffic Safety Section (TRF-TS) director for approval prior to implementation.

Anchor: #i1007142

eGrants Project Grading

Both the State of Texas Uniform Grant Management Standards, Section .36 (b) (8) and Section 18.36 (b) (8) of 49 CFR, Part 18 - Department of Transportation Uniform Administrative Requirements for Grants and Cooperative Agreements to State and Local Governments state:

“Grantees and subgrantees will make awards only to responsible contractors possessing the ability to perform successfully under the terms and conditions of the proposed procurement. Consideration will be given to such matters as contractor integrity, compliance with public policy, record of past performance, and financial and technical resources.”

To comply with the above and to assist TRF-TS in reviewing project accomplishments (performance measures completed, targets achieved), to ascertain whether or not the grant provided a beneficial service to the Traffic Safety Program, and to determine the continued need for a project, TRF-TS has developed the eGrants Project Grading System.

Each subgrantee will receive a grade (A-F) at the end of the grant year based on a possible 100 points. Subgrantees start with 0 points and earn points throughout the grant year. Points will be earned by the timely submission of Performance Reports, RFRs, subgrantee’s performance (per reporting period) and adherance to project budget (per reporting period).

Points will be awarded by the review and grading of Performance Reports and RFRs by the TxDOT project manager. Both the STEP and General grants will be graded on 10 criteria. eGrants automatically calculates and grades five (5) criteria for the STEP grants and three (3) criteria for the General grants. Project managers will review and grade the remaining criteria by reviewing Performance Reports and RFRs and completing the Review and Comment pages of these reports. These criteria include:

    • STEP Indicator, based on grant target number (for Selective Traffic Enforcement Program [STEP] projects only)
    • Matching Contribution
  • Performance Report Submission Dates
    • on time (met the submission deadline)
    • missed the submission deadline
  • RFR Submission Dates
    • on time (met the submission deadline)
    • missed the submission deadline
  • Public Information and Educational (PI&E) Activity Objectives (for STEP Projects only)
  • Project Manager Review of Performance Reports
    • Citations/Arrests for STEP grants/Objectives, applicable activities, and Performance Measures for general grants.
      • Less than Target – Justified
      • Less than Target – Not Justified
      • Meets
      • Exceeds
    • Risk Assessment
      • Termination
      • High Risk Suspension
      • Below
      • Meets
      • Exceeds
    • Overrruns/Underruns
      • Yes
      • No

NOTE: Subgrantees that meet all grant objectives, including submitting all reports within the 30-day time frame, meeting all grant goals and target numbers and staying within the grant budget by 10% through the grant year will receive a score of 90, or an A.

NOTE: The grading process is cumulative and can be viewed at any time by the subgrantee or the project/program manager by clicking “Project Grading” found on the bottom right side of the grant.

TxDOT review includes:

  • The project manager will review Performance Reports and complete the Review Comments page of the Performance Report. This page contains questions concerning the subgrantee’s performance and the project manager’s Risk Assessment of the subgrantee to date. A comments box is also included on the page to allow the project manager the opportunity to provide any comments.

  • The project manager will review RFRs and complete the Review Comments page of the RFR. There are three questions on this page that the project manager must answer concerning budget and supporting documentation submitted by the subgrantee. A comments box is also included on the page to allow the project manager the opportunity to provide any comments.

  • TRF-TS project managers must complete the questions on the Review/Comments pages of the reports to the best of their ability. Each question is answered by selecting the appropriate radio button that best represents the subgrantee’s performance for the reporting period. Therefore, careful review of information contained in the Performance Reports and RFRs, including supporting documentation, is required.

Each Review/Comment page includes a comment box, which is a mandatory field. This is an excellent avenue to document any concerns, reminders, or praise a subgrantee for excellent work. The comments box should be used in conjuction with the project manager’s assessment and grading of the performance period through the use of the radio button selections. The file of record, including a subgrantee’s performance, must be thoroughly documented (especially for a subgrantee that is struggling to meet performance measures/target numbers) to include evidence of continued communication between the project manager and subgrantee.

NOTE: ”Justified” means the subgrantee has included adequate information on the Performance Report Narrative Page explaining difficulties/obstacles encountered by the subgtrantee that directly impacted their ability to meet the objectives and/or activities of the grant for the reporting period. Project managers must use their best judgement, based on the available information, that a valid reason existed that caused the subgrantee to fail to meet one or more objectives or target numbers for the reporting period in order to be “justified.” Valid reasons would have to be considered exceptional and beyond the subgrantee’s control.

NOTE: “Not justified” means insufficient information is included on the Performance Report narrative page to substantiate the reason(s) for failing to meet the grant’s goals and objectives. Therefore, the project manager must choose the “Not meeting target - not justified” radio button on the Comments/Review page.

The eGrants Project Grading Page can be accessed through the eGrants Project Grading link located on the eGrants Help Page. The page provides the latest information concerning the automated grant grading system, including the values of the above criteria.

Anchor: #i1007277

Final Performance Report

Within 30 days of the end of the project period, the subgrantee is required to submit the final Performance Report to the project or program manager (See Chapter 5, Section 2 “ Performance Reports”). This report is intended to be an overall, brief synopsis of the project performance for the year. The final Performance Report:

  • tells whether or not the project objectives were met
  • lists accomplishments (performance measures completed, targets achieved)

After review by the project or program manager, final performance reports are used by The Texas Department of Transportation (TxDOT) Traffic Operations Division - Traffic Safety Section (TRF-TS) for compilation and use in developing the annual report.

Anchor: #i1007317

Final Requests for Reimbursement

Within 45 days of the end of the project, subgrantees must submit all final RFRs to TxDOT. The final RFR is critical to the final closeout of the fiscal year budget, because it allows any residual monies to be carried forward to fund the next year's projects.

Previous page  Next page   Title page