Skip to end of metadata
Go to start of metadata

Background

The "Project Readiness, developed by SUNY Delhi's Department of Campus Information Systems, will be used to assess departmental and technical readiness to undertake a proposed project. The objective is to provide a scoring tool for objective assessments using a set of known and accepted criteria and metrics linked to SUNY CIS's current services and IT vision.  This process will allow for standardized evaluation of proposed projects according to specified criteria (institutional goals, compatibility with current operations and CIS's capacity to successfully implement and administer the proposed project), making decision-making simpler and more transparent.

The Readiness Rubric 

The resulting readiness scores, calculated by the Technology Advisory Group (TAG), can be used to set a sequence of development that minimizes risk and increases the likelihood of meeting the project's goals. CIS resources will be deployed based on the priorities defined by Budgeting and Planning as well as the Cabinet.

The following Rubric is a working document. The final version of the Rubric that the Technology Advisory Group uses for project readiness determination may be found as an attachment to this document.  The Rubric spreadsheet is attached to each Project documentation as part of the project proposal process.

Readiness Criteria

Metrics and Examples

Score

Addresses Current Feedback

How does the proposed project address those issues identified by current users (through the Help Desk/Service Center)? (+5 thru +1 )

 

 

Example: Provide Wonderdesk data showing specific issues related to the project as one of the top five reported by end-users.

 

Within Current Scope Of Services

Is the project within the current scope of services offered by CIS? (+1)
Does the project address a "Market Force Change" seen in a SUNY Delhi functional business unit? (+1)
Does the project address a deficiency in a current Service Level Agreement with a functional business unit? (+1)
Does the project address a technical change in existing systems/services? (+1)

 

 

Example: Cite existing Service Level Agreements between:

  1. a Delhi Functional Unit and a third-party provider,
  2. CIS and a Delhi Functional Unit, or
  3. CIS and a third party provider.

 

Defined Functional Requirements

Has a Product Sponsor from the funcational unit been identified? Please provide name: __________ (+1)
Is there a Confluence page for this project in Confluence? (+1)
Are there user stories for this project (+1/instance)
Are there use cases for this project (+1/instance)
Does the sponsoring functional unit have staff in place (with the skills/training) for implmenting the new service/system? Please identify: __________ (+1)
Does the sponsoring functional unit have staff in place (with the skills/training) for using the new service/system? Please identify: __________ (+1)
Is there a budget for the implementation of this project? (+1)
Please provide funding account number here:__________ (+1)
Is there a budget for the administration of this project? (+1)
Please provide funding account number here:__________ (+1)

 

 

Example: Confluence page should cite specific departmental staff with defined roles and responsibilities.

 

Defined Technical Requirements

Has a CIS Project Manager been identified? Please provide name: __________ (+1)
Are there CIS and departmental staff in place (with the skills/training) for the implementation of this project (This information should be included within the Project Page in Confluence)? (+1)
Are there CIS and departmental staff in place (with the skills/training) for administration of this project (This information should be included within the Project Page in Confluence)? (+1)
Is the technical infrastructure in place for the implementation of this project? (+1)
Is the technical infrastructure in place for the administration of this project? (+1) 

 

 

Example: Confluence page should cite specific CIS staff with defined roles and responsibilities.

 

Aligned with CIS Strategic Vision

Does the project fit into a model of "self service"? (+1)
Does the project fit into a Service Oriented Architecture? (+1)
Will the project enhance current operations? (+1)
Will the project benefit other currently submitted projects? (+1/instance)
Can the project be completed in six months (+1), three months? (+1) 

 

 

Examples:

  • Provide a list of specific manual processes that would be replaced,
  • Include examples of integration and interoperability between existing and proposed systems,
  • Include dependencies and contingencies: projects that require another's completion or progress before it can proceed
  • The functional Unit's business calendar denoting acceptable service schedules

 

Understood Complexity

Does development affect only one service/system (+1)
Does development align with the current functional users' business processes (+1)

 

 

 Example: Provide list of all Functional Units who currently use all systems under consideration.

 

READINESS  RATING


 

Considerations

A recent discussion has evolved questioning the inclusion of campus-wide criteria and metrics (gleaned from the Campus Mission, Vision, Program of Work etc.) as part of an assessment in IT readiness (i.e. project understanding and infrastructural preparedness).

Observation 1 

Confusion exists over the big picture of campus decision making and how the readiness and prioritization rubric fits into that big picture.  The rubric does take institutional strategic questions into account with a point for the existence of the Organizational Analysis as well as points for Delhi's Mission & Strategic Vision and Delhi's Institutional Goals & Long-term Assessment.  However, the points system seems to downplay these elements, which are indicators of whether the project is one we should want to do. The rubric seems to award more of its points to questions of our readiness to take on a particular project. 

Consideration 

The rubric was originally designed to assess CIS's ability to successfully complete the project (on time and on schedule) as well as meet the project's goals of Sponsor (Product Manager), and not as a tool for assessing the overall value of the initiative to the campus. Priorities should not be a CIS decision, hence the involvement of TAG and the Cabinet. CIS should act as facilitators. Ultimately it is up to the Administration as to which projects they would like to see us initiate. It is quite possible that a very valuable project would be beyond the scope of services offered by CIS. In this case the rubric would serve as a valuable tool to identify the gaps in CIS and thus a road map for development. The larger strategic questions about Delhi's mission, vision, and goals are better addressed separately from the readiness rubric.

CIS should work with other core services, e.g. Facilities, to develop unit specific readiness rubrics for proposed projects that would be implemented in conjunction with CIS or even primarily by them.

Observation 2

The first criteria in the rubric asks whether the proposed project will address issues identified by current users through the Service Center.  This question may assume that all projects will be evolutions of existing CIS systems and services?  I could imagine an issue that could be solved through the application of technology, or the modifications of policies, or the addition of human resources, or renovation of a facility, etc.  Many problems that could be solved through technology are not thought of as technology problems by the campus community.  Members of the campus community won't be calling the help desk to ask for a solution to a problem they don't think of as a technology problem.

The rubric will systematically favor projects that resolve issues that have always been associated with CIS, even though CIS may be just as well prepared to resolve issues that have never caused a single call to the help desk. 

CIS is compelled to dedicate resources to existing services, as many functional units on campus may rely on these services to perform their own core business. Highlighting user feedback is an attempt to address issues and deploy technologies to meet current service level agreements before taking on others. This approach also complies with several practices embraced by CIS in the adoption of Agile Methods: user-developers, emergent design, evidence-based decision-making, incremental and iterative development, etc.

CIS will also embrace several levels of engagement: operations, enhancements and projects. Operations will not need to be processed through the Rubric, with an assumption that CIS is already prepared (ready) to support current services. Enhancements and Projects, by definition, will extend the scope of services currently offered by CIS and would therefore require an assessment. Including Service Center calls provides an opportunity to highlight how an enhancement or project may also solve an existing, yet not obviously related, issue.

Readiness Assessments

Current CIS Readiness Ratings

This document is devoted to explaining the reasons for and processes behind the CIS Readiness Rating as part of CIS's overall IT Governance and Project Management practices. The current readiness rating is located here.

References

Project Prioritization, Aligning IT Activities with Client and Institutional Priorities, David Blum, Zane State College, Zanesville, Ohio.

Project Prioritization Scoresheet, David Blum, Zane State College, Zanesville, Ohio.

CIS:Creating thr Right Environment, Becoming a Strategic Partner Ernie Nielsen, Executive Director of Enterprise Project Management for Brigham Young University

Labels
  • None
  1. Jan 29, 2008

    Dear Patrick and Other Contributors,

    I have reviewed the latest on the Readiness Rubric page and I like the direction you've taken things.  The rubric now has a better defined scope that focuses on CIS's readiness instead of half-tackling the issue of campus priorities.  Just to prove that some people can never be satisfied, I'll add the following additional observations: 1)  I don't completely understand the text immediately above the rubric table, which states the following: "The resulting rankings will be used to set the sequence of development.  CIS resources will be deployed based on the priority defined."  What does it mean to "set the sequence of development?"  It seems to imply that projects will be developed in order of descending readiness, but that doesn't quite make sense.  I assume I just don't understand the statement.  Can you be clearer as to what you mean by "the priority defined?"  Since the readiness rubric doesn't define priorities or rank projects by priorities, I assume this sentence is referring to priorities defined elsewhere?2)  In an ideal world a requesting department, organization, or individual would come to CIS with a well defined (all requirements clearly stated) project that enabled CIS to assess readiness and put a rough price tag on the project costs and the on-going operational support costs.  Then campus decision making bodies could weigh benefits to campus mission/goals/objectives and costs of all proposed projects.  That would completely remove CIS from the gatekeeper role and give the appropriate decision makers enough information to make decisions.  I'm wondering whether the existing readiness rubric really provides those campus decision makers with enough information.  How do campus decision makers convert readiness rankings to costs in dollars or schedule?  I fear that the readiness rubric does a good job of ranking different projects by readiness, but that it does so in quantitative terms with no units and therefore very little meaning to campus decision makers.  Let's say one project has a readiness score of 5 and another project has a readiness score of 25.  Maybe that represents a difference of $200,000 and 1 year of schedule.  Maybe that represents a difference of $500 and 1 week of schedule.  As a member of cabinet or some other campus decision making body, how would one know?  If they don't know, how do they make use of the readiness score?3)  The project listed under the "Current CIS Readiness Rating" are scored according to the old readiness rubric before you made the most recent changes. I really appreciate this open process you are using to define the decision making methodology.  I'm also appreciative that you've taken my comments into consideration.  Thank you.Best regards,mark

    1. Jan 30, 2008

      1)  I don't completely understand the text immediately above the rubric table, which states the following: "The resulting rankings will be used to set the sequence of development.  CIS resources will be deployed based on the priority defined."  What does it mean to "set the sequence of development?"

      I have edited the sentence, clarifying (hopefully)the process. The TAG reviews and assesses the projects and the B&P Committee and/or Cabinet prioritizes the work. CIS will take on any project identified by these groups, despite the readiness ranking, however a low readiness ranking should indicate to campus decision-makers the risks associated with undertaking the project: ill-defined requirements will lead to increased costs and time as they are identified through an implement, assess and re-implement process, final functionality/features may not meet expectations, the project may conflict with other services in operations or development, the project may stall due to dependencies not identified at the time project was initiated, etc.

      2) ...How do campus decision makers convert readiness rankings to costs in dollars or schedule?

      As part of the "Defined Requirements" there are two metrics:

      • Is there funding available for the implementation of this project? (+1)
      • Is there funding available for the administration of this project? (+1) 

      In order to assess this (and get the points), the "price tag on the project costs and the on-going operational support costs" must be present, and they would be within the Audience and Organizational Analysis templates that will be used by TAG to assess CIS readiness, as well as the Unit Plan template.

      The Budget and Planning Committee and ultimately the cabinet would make the financial decisions.

      If the campus decision-makers want to spend 200,000 and a year on a project that was deemed a 5, then ultimately that's their prerogative. But hopefully they will recognize that, because it was scored a 5, that the 200,000 is more likely to be 2,000,000 and 1 year, five. Contrast this with a project ranked highly, becuase it was ranked highly, CIS understands and feels confident in implmentation, therefore the estimate of 200,000 is more creditable.

      3. You may have caught me mid-development (now that's the real-time information sharing I am looking for). I took the Excel sheets, originally attached here, and moved them to a newly created page dedicated to just posting the current readiness rankings. They are also updated: the old criteria and scores removed. It is here.

      And finally, this process, where CIS policies, services and systems are shaped by the wisdom of the crowd rather than top-down (leadership vision, committees, etc.) is exactly the direction I would like to move the department. Who better to identify not only the practical issues faced by end-users as the undertake their business operations, but also define better working environments , than the end-users them selves. What Eric Raymond calls "Co-developers," or what Don Tapscott and Anthony Williams site in Wikinomics as "prosumers."

      I actually really appreciate your continued effort, and my challenge coming in new to the organization, is to try and solicit this level of activity throughout the campus.

      First there is the learning curve, that is, what are Agile Methods (the school of thought around what's happening in Confluence): openness/transparency/honesty, self-organized groups, emergent design, iteration and incremental development, etc.

      Then there is the adoption and practice: finding folks who understand the value and will help to create the culture.

      And finally there is the practicality, knowing that I will not be able to flip a switch, folks will need time to understand the processes, get comfortable with the practices, and probably see some results.

      So again, than you for working with us. 

  2. Oct 15, 2008

    The Project Readiness Development Cycle requires documentation by the department or individual initiating the project request. We have developed a template that lists the basic questions that should be answered in that documentation. The template can be found: here. Departments or individuals who are requesting project work should start by the questions listed there. Projects will be listed in the "Open Delhi Discussions" area of Confluence so that the discussion of the projects can be available to the campus community at large. 

  3. Apr 29, 2008

    Anonymous

    I recently attended an information session on the soon-to-be TAG and the readiness rubric.  I think the CIS team is making great strides toward a more rational, participatory, and transparent process.  I congratulate them for that.  Some of my favorite characteristics of the proposed process include the following:

    1)      The rubric encourages sponsoring individuals or groups to think about the important questions and really come to TAG well prepared.

    2)      The process is open to campus scrutiny, so it does not lend itself to "pet" projects that do not stack up against objective measures.

    3)      The process evaluates our ability to successfully implement a project and not just the benefits of a project. 

    Based on the discussion at the information session, I see the following major challenges to getting the full benefits of the proposed process:

    1)      The process relies heavily on a vibrant Confluence community, which will not exist until there is an education effort, which would then have to lead to a minor cultural shift on campus.

    2)      The linkage to the overall campus prioritization and spending decision process is still not well defined.  Until that happens, the proposed process may be limited to essentially an internal CIS process.  That doesn't mean it won't be valuable, but it won't get the CIS department out of their unwanted role as gatekeeper. 

    Thanks to the CIS team for their efforts to date on the proposed process and for their continued efforts to improve and implement the process.

    1. Apr 29, 2008

      Sorry.  I didn't intend to send my comment anonymously.  The April 29, 2008 comment on TAG and readiness rubric is from me. - Mark.

      1. Apr 30, 2008

        Mark,

        Again thanks and I think you are right, both education and overall alignment is not well defined.

        One the first point, this process could take place through traditional communication practices; meetings, emails, draft documents, etc., Confluence is just a tool that increases the productivity. In order to avoid scaring folks its important to highlight how this tool compliments, rather than replaces traditional collaboration methods. To meet the challenge I hope the TAG can serve to compliment CIS' efforts in educating the campus. Also, TAG, CIS and I will need to hold firmly to the development practices, including the use of Confluence. The logic: Delhi needs IT governance and project management, no matter how we choose to undertake these tasks there would be a learning curve for the campus as they engage with the process. So we have a process, let's start teaching.

        The second point is a great concern for me. We need to work with Budget and Planning to ensure our process aligns with their vision for prioritization, and on a more practical note, their own processes. The goal is to provide campus decision-makers (B&P, Cabinet, etc.) with real numbers for costs and time, so they can make more informed decisions regarding development. If we can provide accurate IT resourcing estimates, then I think we are furthering that effort. Ideally we give them a list like...

        Project Name

        Cost

        Time

        Rating

        Widgets

        100K

        3 mos.

        37

        Deelybobs

        100K

        3 mos.

        28

        Gadgets

        20K

        8 mos.

        21

        Dohickies

        ???

        ???

        09

        Thingamajigs

        250K

        2 yrs.

        02

        ..and let the decision-makers decide what is acceptable risk. 

        Thanks,

        Patrick 

  4. Oct 15, 2008

    I am a bit concerned that the organizational and audience analyses are too extensive.  Will potential proposers be turned off by the required data collection and presentation?  Perhaps one goal is to limit frivolous requests, but can the fore mentioned analyses be simplified and shortened?  Do the other questions in the rubric not, more or less, cover the issues of the organizational and audience analyses? 

     Jack

  5. Jun 16, 2010

    I have attached a spreadsheet version of the Readiness Rubric that incorporates some changes and clarifications. This version is designed to be used at the beginning of the assessment phase of projects. A column has been added for responses to each of the rubric elements. Cells tinted light blue are for CIS responses. Other cells are for the Sponsoring Department to complete with supporting information. In a large project the responses might be links to documentation within Confluence.

  6. Jun 10, 2009

    In scoring projects with the readiness rubric (and presumably developng a proposal), it would be helpful to have definitions and exampls of the following terms available:

    Market Force Change

    Functional Business Unit

    Service Level Agreement

    Future Business Practices

    Sponsoring Unit

    Model of "Self Service"

    Service Oriented Architecture

  7. Jun 10, 2009

    Hi Jack,

    This was also a topic of conversation at the last meeting. Let me take a stab at defining these terms:

    Market Force Change an external change that impacts how the College provides services or communicates. An excellent example of responding to a Market Force Change would be the investment in the Nursing program in response to the national shortage of healthcare workers. Other Market Force Changes include Enrollment Services use of Instant Messaging to communicate with potential students, Alumni's use of Facebook and Twitter.

    Functional Business Unit is our terminology for the departments and divisions of the College. Enrollment Services, the Technologies Division and CIS are all Functional Business Units

    Service Level Agreement is (or should be) the agreement CIS has with its clients about the extent of services and availability. We offer Moodle 24 hours a day, 7 days a week, but we provide technical support for it 8 hours per day, 5 days per week. If there was a clamor for supporting Moodle 16 hours a day to support students in California and Hawaii, a project with an external service provider might address our local service deficiency.

    Future Business Practices when we ask about Future Business Practices it is to assess whether the requesting department has thought about how implementing this technology will change how the department does business. Too often our clients look at implementing technology as a solution to their problems, when in reality the technology is just a tool, not the solution. Often the technology has requirements for the department's staff time to maintain it, records to keep or funds to upgrade it. We want to make sure the department understands what it is getting into when the glitter fades.

    Sponsoring Unit is the department requesting the project. The "Project Sponsor" is usually the lead project team member of the department requesting the project. The "Project Manager" is the lead project team member from CIS implementing the project.

    Model of "Self Service" CIS is trying to empower its users to run their own reports, queries and services. We want to encourage projects that help our clients meet their needs without our intervention. It is a small investment we make that pays off later with less use of our staff time and clients that have a better understanding of their needs.

    Service Oriented Architecture will take more than a paragraph. Let me refer you to this Wikipedia article. Also see our Interesting Article section for two articles posted on 5/27/2009 regarding SOA.

    Thank you for the question, Jack. It is important that CIS and TAG, as well as the campus community, have a common understanding of the terms in use.

    Ron