Request For Proposal Evaluation Criteria Manager
Managing
1. Evaluation Criteria
2. Request For Proposal Issues
3. Vendor Answers to the Request For Proposal
4. Questions to the Vendor
5. Scoring of Vendor Proposals to the RFP
6. Determining The Vendors that Meet or Exceed Your Criteria
Table of Contents
Introduction 1
The IT Acquisition Process 1
Steps in the IT Acquisition Process (The MAP(tm)) 2
Form the Team 2
Establish the Decision Criteria 2
Determine the Relationship with the Vendor 3
Gain Management Approval 3
Develop the Contract 3
Issue the RFP 3
Conduct the Bidders Conference 4
Evaluate Potential Suppliers 4
Implement the Zone of Consideration 4
Administer the Contract 4
Why would we want to do this instead of meeting and discussing the issue 5
The Basics of the Evaluation Criteria Manager Application 6
How does it work? 6
The Project Requirements and Specifications (The Issues) 8
The Evaluation Criteria 11
Importing the Requirements and Specifications - The Issues 13
Format The Issues So They Can Be Read Into The System 13
The Switchboard 15
Required Steps to set up a project 16
The Add Employee Screen 16
The Rating Team and the Scoring Team 17
The Project Entry Screen 18
Set up Task Information (Evaluation Criteria Summary Topics) 21
Import Business Issues (RFP requirements, Contract Issues, etc) 22
Importing the Issues 22
Assign Major/Minor Tasks to Issues/Requirements Set business Issue Imp. 25
Assigning Categories to the Business Issues 25
Set Business Issue Importance 27
Set Project Evaluation Criteria (Define Percentages Etc,) 28
Generate Project Major/Minor List for Evaluation Criteria Report 30
Changing the Evaluation Criteria Percentages 31
Enter Vendor Information 32
Import Vendor Proposal 34
Reading the Vendor Proposal into the Evaluation Criteria Manager 34
Proposal Scoring - Enter Scores and Notes 36
Reports 37
File Structure and Setup 38
The Main Access Application File 39
The Company Database Tables 41
The Project Database Tables 42
Linking the Company and Project Database Modules to the Main Application 43
A Test Project 46
The RFP Process - A Step By Step Checklist 50
REQUEST FOR PROPOSAL PROCEDURES 50
Introduction 50
TYPES OF PROCUREMENT 51
Determine Criteria 51
Guidelines 51
SAMPLE DOCUMENTATION
The Evaluation Criteria Manager software was built on the foundation that:
Once scores have been determined a simple method for determining the proposal to select is to divide the vendor’s project cost by the vendor’s final score giving a dollars per point. This price/performance ratio will show the best value proposal.
Some companies will attempt to balance the costs and set a point value to the cost, then adding the resulting points to the vendor’s final score. This method causes the vendors to be rated based on another vendors cost submission and does not really evaluate a vendor based on only what they submitted and how that submission meets the needs of the company.
The price performance ratio, however, aligns the vendors based on the quality of the proposal and that proposal’s costs for implementation. It allows each proposal to be graded on its own merits and eliminates the result from being tarnished by another vendor’s submission.
The software requires the issues to be read into the database, evaluated by the company for importance and then summarized into a report showing the weighted relative importance of the issues. Comments by individuals rating the issues can be input into the database and a report of the comments can be printed.
The software then provides for the input of vendor proposal information cross-matched to each business issue in the RFP that was read into the database.
Vendor response can be scored and comments input by each member of the evaluation team. Specific questions to the vendor can also be included and a report, in letter form, can be printed with the specific questions as they apply to the particular RFP issue.
Once scoring is completed the software then produces a report showing the vendors final weighted total score based on a raw averaged team score for each issue and the weight applied to each issue.
The main advantage of the software is to allow individual team members, either those rating issues or scoring proposals, to rate and score as time permits rather than eating up meeting time. Also since the rating and scoring is performed individually and independently from group dynamics as well as being rated as scored on the computer the software captures the comments and questions on the fly rather that relying on handwritten notes that may become lost or unreadable.
RFP issues written in an RFP are the complete set of requirements, specifications, and contractual obligations that, the company has determined, are the minimum standards acceptable for a contract to be awarded.
There may be well up to or over a thousand individual issues that need to be tracked.
Issues such as:
System-Wide Requirements. |
1.1. Following are mandatory requirements: |
1.1.1. All parts of the system must be integrated with a minimum of data redundancy. |
1.1.2. The system must have a facility for recording all transactions, and the status of each transaction, to facilitate recovery in the event of disaster. |
1.1.3. The system must be capable of performing mass file maintenance based on changes in profit margins, shipping charges, cost of durable goods or any other field which has a standard value |
1.1.4. The system must provide for purging of data from the system with a full accounting. |
1.1.5. Documentation and operating procedures (technical and user) must be supplied. |
1.1.6. The security and access to the system and its data must be assured. |
1.1.7. Authorization levels (e.g., read, add, change, delete by application, file, record, field) must be assigned by a designated security officer. |
1.1.8. Security audit trail reports must be provided to both identify authorized individuals accessing and entering and/or changing data in the system, and to list the transactions. |
1.1.9. Vendor personnel must have expertise in the Customer business. |
1.1.10. Vendor must provide the hardware, or additional hardware, required to run the system at the Customers site. |
The issues form the basis of an evaluation criteria
A typical Evaluation Criteria may look like the following:
I. EXECUTIVE SUMMARY TOTAL: 500
A. Completeness.................................................... 100
B. Understanding of Current Operations................. 200
C. Understanding of Requirements.......................... 200
II. PRESENTATION OF RESPONSE TOTAL: 1,000
A. Completeness.................................................... 200
B. Clarity of presentation........................................ 200
C. Organization of presentation............................... 200
D. Understanding of DOL objectives...................... 200
E. Qualified acceptance of terms and conditions...... 200
III. EXPERIENCE TOTAL: 3,500
A. Client reference............................................... 1000
B. Qualifications and Special Experience
1. Sufficient staff depth..................................... 100
2. Software expertise....................................... 200
3. Database expertise....................................... 200
4. Software integration..................................... 200
5. Database design........................................... 100
6. Windows expertise...................................... 200
...................................................... subtotal: 1,000
C. Project Management
1. Project management experience.................... 400
2. Education..................................................... 100
3. Technologies................................................ 250
....................................................... subtotal: 750
D. Staffing
1. Related experience....................................... 300
2. Education..................................................... 100
3. Technologies................................................ 350
.................................................. subtotal: 750
IV. FINANCIAL TOTAL: 2,000
A. Financial Statement............................................ 500
B. Evidence of financial solvency.......................... 1500
V. OVERALL PROJECT APPROACH TOTAL: 3,000
A. Schedule/Project Plan...................................... 1500
B. Overall Plan Realistic....................................... 1500
C. Bonus for completion by X/X/XX...................... 500
TOTAL: 10,000
This Evaluation Report example has five (5) main categories and multiple subcategories.
The issues can be summarized into a few main elements such as:
For example:
“1.1.9. Vendor personnel must have expertise in the Customer business.” could be summarized under “Company Background” or “Client References” or “Critical Success Factors”, etc. Where it goes depends on the particular issue and the main area that the company decides it logically fits.
In addition there may be several issues where the RFP asks for a response such as “Company Background” or “Client References” or “Critical Success Factors”. These particular issues may have a common link like “References” or “Vendor Resumes” or Vendor employee Expertise” or Vendor Company Expertise”, Etc.
The Software allows any issue to be placed in any category that the company decides it belongs. Many issues may be placed into the same category. A category could also have only one issue. There is no limitation on where issues are placed.
The software will combine issues which have matching main categories and sub categories.
The resulting summary will look something like:
EVALUATION CRITERIA SAMPLE REPORT.
Evaluation Criteria
Project ID Major Task Description
MinorTask Description Section
SAMPLE Example of a Summary
OT Other Terms and Conditions
7 Completeness of response 1.1.9.
CB Company Background
12 Extent of Supplier’s experience and longevity in the Industry
67 Financial Condition
ES Executive Summary
7 Completeness of response
31 Understanding of Current Operations
36 Understanding of Requirement
ETC.
A typical evaluation criteria will have points assigned to categories and subcategories.
Usually there may be about ten or so main categories like the Executive Summary, Experience, References, Project Presentation, Financial Stability, etc. Within each Main category there may be one or more sub categories some of which may have the same title.
The Main categories should have a percentage weight assigned to each one. The total percentage of the Main categories will add up to 100%.
Then a total point value is assigned to the project (10,000 points) and the points are distributed to the main categories based on the percentages giving relative weights.
The Software will distribute each category’s points to the related subcategory based on the overall importance of the subcategory within each main category.
The Evaluation Criteria Manager allows several ways to get the Specifications and Requirements into the system. Input can be by referencing a Word document that has the Requirements listed or using the Access wizard to import text into a table. First we will go through the process for using the Access Wizard and then go through the process for importing direct from MS Word.
NOTE: The Evaluation Criteria Manager requires that the imported text be in an outline format.
While it is recommended that every line have an outline, the software will assign an outline number to any text line missing an outline. The software can interpret what is an outline and what is not. So, if a line has outline 1.1. and the next line starts with a letter not followed by a period it will assign an outline number 1.1.P1. to that line. P1 is the first paragraph after 1.1. but with no outline number. If there was a second line without an outline number it would be assigned 1.1.P2. etc.
If the first outline number was say A. the software would add numbers until it reads a B. and then start over.
Importing Using Text and Access Wizard
Please note that each issue should have a reference number, outline number or some other means of identifying the issue in the database. IN all cases the first line MUST have an outline code e.g. 1. or A. or a.1. etc.
Also, the reference number SHOULD be concluded with a period and a space (“. ”). The space is critical for the software to separate the outline numbers/letters/roman numerals outline designators from the descriptive text.
If the first line is missing an outline number the software will take the first text string up to the first space and use that as the outline number.
For example the last item (1.1.9. Vendor personnel…) will be split to two fields “1.1.9.” and “Vendor personnel…”.
Without the period and space after the 1.1.9, the ID will be 1.1.9.Vendor and the descriptive text would be “personnel must have expertise in the Customer business”.
The issues must be cut/pasted/written into a text file with each issue preceded by its unique reference number ID. The reference number and its the descriptive text MUST then be followed by a line break. This is required to make sure each issue is imported into the database as a separate record.
Write down the saved file name and the directory path since you will need this when reading the file into the Evaluation Criteria Manager.
All special characters such as tabs, paragraph marks, line breaks (except the line break at the end of each description), page breaks etc. internal to a single issue must be removed.
Cutting and pasting the RFP issues into a single document may seem like a daunting task but it does take a relatively short time.
Once structured the document should be saved for input into the system.
There are some inconsistencies with text input into Access. While most text will read in fine occasionally some issues my not be imported. Going back over the text document, there may be unprintable special characters that refuse to go away. In some cases theses special characters can prevent the issue or the next issue to be skipped when importing into the Evaluation Criteria Manager.
In this case it may help to highlight all the issues in the saved text file and copy them into an Excel Spreadsheet, review the Excel document and then save the Excel document as text. It may seem strange but it has worked on the difficult errors.
Now the RFP issues are ready to be input into the system.
Importing Using Built in Forms
As in the previous example the Specifications and requirements need to have an outline number. In Word this is easy by assigning an outline format to the document text. This software will read in the word document convert selected paragraphs to text and use the associated outline scheme to the imported text. Any outline scheme will work, numbers letters, roman numerals lower case or upper case.
This import is done through the Switchboard menu.
The Switchboard is used as a navigation tool to allow a logical progression through the RFP Evaluation process.
Before reading the issues the system will require an employee (the Project Manager) and a valid Project ID. A valid Project ID is simply entered from opening the project entry screen and entering a project name/ID.
Opening the Evaluation Criteria Manager switchboard, position the mouse arrow on the Employee Information button and left click the mouse.
Right clicking the mouse button will display a help message for this button. Make sure the Microsoft Office Assistant is activated to use this feature.
On the Employee Information Screen Click Add/Edit Employee to System.
Enter an Employee Name and Employee ID. The ID should be fairly descriptive with initials, first name and first initial of the last name or first name initial and last name. It is fairly free form but each employee must have a unique ID.
The Project Manager can be any employee in the Database.
When entering employees to be raters and/or scorers there is a maximum of 12 raters and 12 scorers. They can be the same or different employees since rating and scoring are separate areas in the system.
This is because of the reports and how many can be listed in a report column with totals/deviations/ and class etc.
I have usually found that teams that exceed 8 usually take longer to complete their tasks in an inverse proportion to the number of members.
The Rating Team and the Scoring Team
To set up the team for rating the issues go back to the Main switchboard and position the mouse arrow on the Add/Edit Employee(s) to Project Team (Must Have a Project Defined) button and left click the mouse.
Select the ProjectID and the Project Manager Name and Project description will display.
Then select the arrow under “PROJECT TASK RATING TEAM” to select employees who will be rating the issues.
OR
Select the arrow under “PROJECT TASK SCORING TEAM” to select employees who will be scoring the proposals.
Team members can be added at any time.
Team members can be removed BUT any members score either in the rating of issues or scoring of proposals will be included in the evaluation criteria report and the vendor scoring report.
