Third-Party Evaluation Services Request for Proposal

Transcription

1 Third-Party Evaluation Services Request for Proposal RESPONDENTS ORIENTATION November 5, :00 4:00 PM

2 Welcome and Introductions Who s in the room? Name and organization

3 Request for Proposal (RFP) Ø Procure the services of a third-party evaluation team that will lead the development of the Evaluation Research Design, Implementation Study, RCT Impact Study, and Cost-Effectiveness Study in collaboration with SDWP and the National Evaluation Coordinator selected by the DOL for the Workforce Innovation Fund projects. Ø Terms of contract Eight month term: Feb. 1, 2015 Sept. 30, 2015 Based on the DOL acceptance and approval of the Evaluation Research Design plan, need, performance, and funding availability, this agreement will extend through September 30, 2019 for a total period of performance of 56 months.

4 Important Dates Ø Notice of Intent to Submit a Proposal due November 7, 2014, 5:00 pm (PST) Ø Statement of Qualifications due November 14, 2014, 5:00 pm (PST) Ø Proposal due November 20, 2014, 3:00 pm (PST) Section 1.4

5 Who We Are The San Diego Workforce Partnership Ø Mission: To empower job seekers to meet the current and future workforce needs of employers in San Diego County Ø Vision: Every business in our region has access to a skilled workforce and every job seeker has access to meaningful employment. Our community will view us as the leader for innovative workforce solutions, which add great value to our region Section 2.1

6 San Diego Consortium Policy Board The Policy Board appoints Workforce Investment Board members and is responsible for overseeing WIA funding and policy development. San Diego Workforce Investment Board Federally mandated board which shared responsibility with the Policy Board for overseeing funding and policy development under the WIA Adult Programs Committee Provides oversight and direction to the Americas Job Centers of California and adult programs. Youth Council Oversees the design of local youth services and activities. San Diego Workforce Partnership Board of Directors Represents an array of San Diego s industries, Provides oversight of SDWP Operations. STEWARDSHIP Every funded program is approved through a strategic system of boards and committees. Section 2.4

7 Workforce Innovation Fund Ø What is the Workforce Innovation Fund? Ø WIF Guiding Principles Section 2.2

8 Breaking Barriers in San Diego On September 29 th SDWP was awarded $6 million from DOL to replicate and extend a small pilot from Ramsey County, MN. The overarching research questions related to this program design include: Ø Are the effects identified in Ramsey County replicable amidst a larger sample size in a different county? Ø How are participants barriers to employment related to success in the program? Ø What types of outcomes can be expected when TANF recipients with a disability exemption receive the IPS model? Section 3.1

9 Logic Model Section 3.1

10 WIF Program Design Background Ø TANF population with disabilityrelated exemptions Ø Individual Placement and Support models Ø Supportive Employment model Section 3.1

11 Key Project Deliverables/Requirements Year 1: February 1, 2015 September 30, Submit Ini+al Evalua+on Research Design (ERD) report to SDWP 4/1/ Final ERD, final performance data template and final evalua+on budget 6/1/2015 that includes IRB approval to SDWP 3 Tool for random assignment of approximately 1000 par+cipant families 6/1/ Technical assistance/support genng ERD approved by DOL/IRB 9/1/2015 Years 2-5: October 1, 2015 September 30, 2019* 5 Interim Annual Report #1 ** 9/30/ Interim Annual Report #2** 9/30/ Interim Annual Report #3** 9/30/ Final Report** 9/30/2019 Ongoing Requirements/Deliverables 9 Monthly Status Reports and check- in conference calls with SDWP staff Monthly 10 Project kick- off mee+ng in San Diego and other in person mee+ngs as Frequency as determined needed for successful comple+on of project. by Project Plan 11 Training and technical assistance on research design to program staff Dates as determined by the Project Plan *Deliverables and related funding contingent on Year 1 performance and acceptance of ERD by DOL (Deliverable #4). **All interim annual and final reports must include, at a minimum, separate stand-alone sections for 1) Implementation study; 2) Randomized Control Trial (RCT) Impact Study; and 3) Cost-Effectiveness Study. More information about the contents of these studies can be found in Section 4.7 of this RFP. Section 3.3

12 Evaluation Research Design (ERD) Ø Primary deliverable for Phase I Ø What is the innovation? Ø How will the evaluation build on the existing evidence base? Section 4.7.1

13 Key Components of Study Responses Ø Approach to developing and finalizing study Ø Importance, purpose and scope Ø Sample research questions Ø Data collection and analysis Ø How you would address potential challenges? Section 4.7.1

14 Implementation Study Ø What is happening and why? Ø Compare program to program design Ø Services Ø Administration Ø Operations Ø Outcomes Ø Provide context for evaluation results Section 4.7.2

15 Impact Evaluation Ø What impact did the intervention have on the treatment group? Approximately 1,000 participants randomly assigned Evaluator to follow control group for duration of service provision Ø Quantify program outputs & outcomes Ø How did participants change over time? Section PHOTOGRAPHY BY MICHAEL SCHUERMAN

16 Cost Effectiveness Analysis Ø What is the cost, per unit, of the results obtained? Ø Was the cost reasonable? Section 4.7.4

17 EVALUATION CRITERIA Written Proposal Scoring Section 4:6: Organizational Capacity and Experience (25 points total) 4.6.2: Organizational Capacity and Resources : Project Experience and Qualifications 15 Title Goes Here 4.6.3: Project Team 5 Section 4.7: Approach to Evaluation Design & Implementation (50 points total) 4.7.1: Approach to Evaluation Research Design (ERD) for RCT Study : Approach to Implementation Study : Approach to RCT Impact Study : Approach to Cost Effectiveness Study : Project Work Plan 5 Section 4.8: Proposed Budget (10 points total) 4.8 Proposed Budget 10 Written Proposal Total Available Points 85 Oral Interview Scoring (Respondents that receive 70+ points on written portion will advance to the Oral Interview Oral Interviews Total Available Points 15 Section 6.2 Total Available Points 100

18 Estimated Funds Available Period of Performance Total Funding Available February 1, 2015 September 30, 2015 $149,428 October 1, 2015 September 30, 2019* $796,950 Overall Funding Total $946,378 *Funding from 10/1/2015 9/30/2019 is based on the DOL acceptance and approval of the ERD, need, performance, and funding availability. Section 3.2

19 Proposal Requirements Ø One original and seven copies (7) total plus a PDF copy on a USB flash drive Single-sided pages Double-spaced No Binders, copy ready format Ø Page Limits Executive Summary 3 pages Proposal Content 20 pages Budget Narrative 6 pages Attachments 10 pages (total) Section 4.0

20 Proposed Budget & Financial Requirements Ø Financial Management Structure (Attachment K) Ø Budget Shell (Attachment F) Ø Budget Narrative Form (Attachment G) Ø Budget Instructions (Attachment J)

21 Financial Management Structure Ø Attachment K of the RFP Ø Evaluation of experience in managing and accounting for State and Federal funds Ø Self monitoring for financial performance and compliance Ø To be completed by the lead agency Section 5.1

22 Budget Instructions Ø Attachment J of the RFP Ø Provides instructions for Budget Shell and Budget Narrative Section J PHOTOGRAPHY BY MICHAEL SCHUERMAN

23 Budget Instructions Ø For-Profit Organizations Negotiated profit line item will not be negotiated at a rate higher than 10% of a respondent s budget. Several factors used in negotiating profit section 5.2 Section 5.2

24 Budget Information Ø Budget Shell (Attachment F) includes: Project Budget Detail Personnel Detail Non-personnel costs Deliverables Activities Section 5.2

25 Budget Narrative Ø Attachment G of the RFP Ø Narrative should include purpose and benefit of the expense Ø Provide calculations (unit cost) when needed cost driver and cost of the driver Ø Not to exceed 6 pages Section 5.2

26 Payment Structure Ø Cost reimbursement contract with full recovery of costs contingent on meeting all deliverables Ø If Contractor fails to meet deliverables, payment forfeiture ranging up to 10% of the total contract cost will be applied Ø Any revisions made to the proposed deliverables must be negotiated and approved by SDWP Section 5.3

27 Conflict of Interest Title Goes Here Ø Appeals Process Ø Cooling Off Period Ø Ex-Parte Communication

28 Appeals Process Ø Written letter to the Manager of Administrative Services Specific evidence for appeal Specific relief sought Ø Appeal must be received within 3 business days from date agenda is posted to SDWP website Ø Third-party appeal review panel 10 business days to determine if criteria for a valid appeal have been met. Section 6.4

29 Cooling Off Period SDWP, the WIB and the Policy Board will not approve or contract with, and will reject any bid or proposal submitted by, an individual or entity who within the preceding twelve (12) months, was themselves or employs anyone who: Ø Is a current, dismissed, separated, or formerly employed person of the San Diego Workforce Partnership, Inc. and Was employed in any position(s) of substantial responsibility in the area of service to be performed by the contract; or Participated in any way in the negotiations, transactions, planning, arrangements, or any part of the decision making process relevant to the proposed contract/service agreement, or was or is employed in a role of substantial responsibility in the same general subject area as the proposed contract; or Is an owner, officer, principal, partner, or major shareholder of the proposed contractor. Section 6.7.2

30 Ex-Parte Communication Ø Restricts RFP respondents from communicating with any SDWP board members/staff on RFP related matters. Ø Ex-parte Communication Conflict of Interest Disclosure Form (required form) Section 6.7.1

31 Proposal Evaluation Process Title Goes Here Proposal due November 20, 2014, 3:00 pm (PST) Selection/Evaluation Process Nov. 20 Dec. 10, 2014

32 RFP Questions Title Goes Here Thank you! PHOTOGRAPHY BY MICHAEL SCHUERMAN