If you have been following along with some of my blogs, you will notice a recurring topic: REQUIREMENTS (Are Requirements Necessary When Purchasing IT Products , What Does Your System Require?).  I know I have written this before, but having good requirements is the foundation of any successful system. If you write really good requirements at the beginning of a project, you will have useful Test Plan by the end of it.


Requirements Development and Testing
In the System Development Life Cycle (SDLC), there are two areas that tend to be short changed, in both the time needed and details required:  requirements development and testing. These are the same two areas that cause the greatest issues if there is no commitment. If well written and easily understandable requirements are created at the beginning of the project, then the developers can build a system that meets those requirements and then those same requirements can become a clear and concise Test Plan. This will result in a Test Plan that thoroughly evaluates the requirements, thereby providing the client with a fully tested system that meets their needs.


Systems Development Life Cycle - Image Credit - Netlocity

Systems Development Life Cycle – Image Credit – Netlocity

Elements of a Test Plan
A Test Plan, in its simplest form, is a chart. Applied Research and Technical Services (ARTS) uses an Excel spreadsheet that includes the requirements, arranged in numerical order, and incorporates columns that let the Tester denote whether or not the system performed correctly.  For example, the requirement may be, “The System shall include a log in screen that contains both ’Username’ and ’Password’ fields.” If during testing, you go to log into the System and either the “Username” or “Password” field is missing, the requirement was not met and the failure will be indicated on the Test Plan.


Testing is an important part of any system development process; however, even with the best Test Plan and the most conscientious Testers, there is almost always some bug that no one anticipated.  That’s ok! Between the unit testing (testing done by the developers during development), using the Test Plan that is based on the requirements,  and the testing being completed by the client (User Acceptance Testing – UAT), most major issues should be discovered and corrected prior to the system “going live.”



A while back I wrote a blog about writing a Request for Proposal or RFP (“How do you write a RFP?”).  That blog discussed the overall RFP creation process. Now, it is time to look at one very specific but important element of the document, Vendor’s Minimum Qualifications.


What are Minimum Qualifications?
A simple answer is the lowest required level of knowledge and abilities that a company as a whole or any proposed individual must possess to be considered for award of the RFP. Carefully crafted minimum qualifications allow the organization issuing the RFP to provide potential vendors with a checklist of a vendor’s “must haves.” The tricky part comes in writing clear and understandable minimum qualifications.


How do you write GOOD Minimum Qualifications?     
Ahh, here’s the rub.  What may be clear to you and your organization may not be clear to a potential vendor. For example, it may be important to your organization that the vendor possesses five years development experience. Writing a minimum qualification as, “Vendor shall have five years IT development experience” leaves the interpretation of the minimum qualification wide open. A better way to write the minimum qualification is, “Vendor shall have a five consecutive years developing inventory tracking programs.” Even though your RFP may be requesting an inventory tracking program, being specific when developing minimum qualifications helps to “weed out” those vendors that do not possess the skills and experience that you require. Minimum qualifications should encompass specific skills and abilities that your organization feels are “non-negotiable” and required in order to complete the project on time and on budget.


This also extends to specific personnel that you want to be a part of the project.  To request that a “Vendor shall provide a project manager with telephony experience” is not a good minimum qualification.  In order for the vendor to propose a project manager that will truly meet your needs, it would be better to write the minimum qualification as “Vendor shall provide a project manager with a minimum of seven years’ experience managing telephony projects for at least 200 users. ”  A vendor, when preparing its response to the RFP may have a great Project Manager in mind; however if that person has never handled a project on the size and scope that matches your RFP, then you will not receive the skill set needed to complete the project. A good minimum qualification will help them align their proposed solution with your true need.


What if there are qualifications that would be beneficial, but aren’t required? 
In addition to minimum qualifications, another level that can be included is the “Additional Qualifications.” These qualifications, for example, may be specific certifications, such as a Project Management Program certification or PMP or Cisco Certification. “Additional Qualifications” are just that, extra qualifications. If a qualification for either the vendor or its proposed personnel is absolutely required then it is a minimum qualification and needs to be portrayed as such in your RFP.



One of the Division of Innovation and Applied Research’s strengths is its highly skilled staff. Another strength is its ability to call upon Towson University’s multitude of talented and knowledgeable faculty. Our latest project involved working with the Maryland State Department of Education’s Division of Early Childhood Development (DECD) and Dr. Nhung T. Nguyen, an Associate Professor in the College of Business and Economics.


DECD contacted the Division of Innovation and Applied Research with regard to a compensation and classification project. Bill Hansman took on the role of both analyst and project manager, and then reached out to the University for a subject matter expert to be part of the project team. Dr. Nguyen, who earned her PhD in Management from the Virginia Commonwealth University, was a perfect fit since compensation management is one of her areas of expertise.


The team decided to use in-person interviews, and surveys, to meet the projects goal of analyzing the job duties at DECD. The team accomplished this by:

  • Analyzing current job descriptions, which included reviewing the State’s MS-22 language and comparing that language with information that was received from the interviews and surveys.
  • Comparing each employee’s resume with the knowledge and skills needed for his/her current position, and determining if the resume indicated if he/she was over or under qualified for that position.
  • Reviewing the internal job structure from the data received, which allowed the team to make recommendations about the DECD positions as they currently exist, and to revise current position descriptions to reflect the work actually being done, as well as to draft new position descriptions. These actions are needed to fill current vacancies within DECD.

Bill and Dr. Nguyen accomplished the project’s goal of analyzing the job duties in terms of scope of responsibility and its distribution across the DECD’s organization units. All of the work the team performed lead to recommending areas for possible changes that would optimize DEDC’s efficiency and effectiveness.


This was a rewarding project for the Division and client and also a great opportunity to develop a new collaborative relationship with one of Towson University’s expert faculty members.



So you don’t think you need requirements when buying a new IT product? If you are buying something for home use, chances are you don’t.  But if you are talking about business use – that can be an entirely different story.


In most professional environments there is more than one user, or more than one person that needs results based on the product.  With several people needing the product to perform in a certain way, it is better to find out how it will be used up front.  (See “What do you require?”).  It takes careful planning and requirements gathering to purchase, install, and customize a  product that meets a specific business need. So what happens if you don’t gather requirements before making a purchase?


Time and money are two words bantered around a lot. So if creating requirements costs time and money, how can it save both?

  1. If you know what your requirements are before you go looking for the software, you will save time when searching for the right product.
  2. If a commercial off-the-shelf ( COTS) product meets all major needs with only minor tweaking, and you already know which requirements are “must haves” and which ones are “nice to have,” you save money!  If the COTS product you find does not meet your needs then you will need to either change your expectations, or spend additional money retrofitting it.
  3. If you spend more time upfront determining all requirements, including requirements on where the product will be housed and who will need access, less time and money will be spent on implementation.
  4. If you spend the time and money on the requirements up front, you can then use the information in a Request for Proposal, ensuring contractors understand what is needed, so they can provide an accurate cost estimate, once again saving time and money.

So what is the “worst case scenario” if you don’t obtain your requirements upfront?



  1. Purchase a product that doesn’t meet your needs and you do not use it.
  2. Spend time (in some cases years) trying to make or modify the product to work.
  3. Lose institutional information, which is never documented and cannot be retrieved if one person holds all the knowledge.

Requirements gathering can be onerous, but if done correctly they can be worth their weight in time and money!



Last week I attended a Return on Investment (ROI) workshop by the ROI Institute, hosted by the International Society for Performance Improvement, Potomac Chapter.  Based on my professional background, the lens I used to understand the content and discussions, was training.  As I reflected afterwards, I was left thinking that we, as trainers and developers, really could do a better job demonstrating the value of what we do.  Often times when determining if a training has been a success or not, the only data we draw from is class evaluations, and while evaluations can shed light on trainee reaction and learning, it doesn’t measure success.  I think we can do better.  As trainers, we need to start asking the bigger questions:  Are trainees using the information?  How is this training impacting business?  Has it increased productivity? Decreased re-work? Decreased citizen complaints?


Sometimes the biggest obstacle to answering these questions is being able to identify, from the analysis stage of training development, what is the goal of the training?  It is unlikely, for example, that the goal is simply for trainees to learn a new procedure.  That procedure was not created for the sake of creating a procedure; instead it was created to accomplish a task that works towards a specific goal (increased productivity, decreased errors etc.).  Therefore, the question cannot merely be, did trainees learn the new procedure?  It should be, instead, did trainees implement what they learned and produce some measurable benefit?

Some of the tips Dr. Jack Phillips (ROI Institute) presented were the following:

  • Do not start training development by identifying learning needs, instead, start by identifying the ROI objectives, such as, what are the business needs? Then, build you can begin building your learning objectives.
  • Make your ROI objectives specific and measurable, for example: Decrease accidents by 5% in four months after training has been implemented.
  • Do not wait until the end of a training event to evaluate, you should be evaluating at every stage of the process.




In past blog posts I’ve introduced what CAIRE is doing to support RTTT and discussed the how we are evaluating the implementation and utilization of the projects.  In this post I wanted to share the perspective of a “freshly minted” graduate who has recently joined the CAIRE team.

Jessica Lake

Jessica Lake

Jessica Lake, RESI Research Assistant, recently graduated with a degree in Sociology from Brown University.


What was your first impression of CAIRE’s work?
When I began working on CAIRE, the initial thought of evaluating 54 projects was a bit overwhelming. However, as I’ve spent more time working in-depth on each project, I’ve found that the diversity of topics has made my job more intriguing and has allowed me to develop a higher degree of workplace versatility. Although all projects relate to the broad topic of education reform, each one has a unique purpose and intended impact on the education system. Some projects are focused on broad data systems, others are geared towards aiding educators, some are specifically targets towards students, and so forth. As such, my work has provided me with a broad range of insight, knowledge, and perspectives on the field of education that I could not have gained without such variety.


How are you using what you learned in the classroom?
During my time working with CAIRE, I have relied upon many of the skills that I developed during my undergraduate education. My abilities related to document analysis: formal writing, group work, and conducting research have all contributed to my effective performance of daily job tasks. Additionally, as a Sociology major, I’ve been able to apply some of the academic knowledge I gained to my work and have found real world value in the abstract ideas and concepts from my classes.  Overall, it’s proven that my undergraduate experience was absolutely beneficial for my future career.


What is it like being a part of the CAIRE team?
Although I had internships in various offices during college, they provided only limited time working with professionals because they were part-time and temporary. Now that I’m out of college and working with CAIRE, I’ve found that my collaboration with experienced staff has been enriching and beneficial. Everyone working with CAIRE has many years of experience and, thus, has a great amount of knowledge and advice to share. I’ve not only learned about specific workplace skills and duties, but also about the  importance of strong relationships amongst co-workers,  the ways in which a wide variety of perspectives can  impact a project, and how to solve problems in real life settings. Overall, everything I’m learning from my co- workers is helping me build a great foundation for my years in graduate school and my future career.


When we first started putting this blog together, I was curious to see what Jess would have to say about her work experience so far.  After reading her comments, I’m even more encouraged to know she is not only using the knowledge she gained in college but is enjoying her experience and also learning tons more along the way.



As you know, I enjoy learning and sharing new things. My latest tech adventure involves creating a series of tutorials to assist you, our clients and partners, with key principles used to optimize business and organizational processes. These tutorials will be housed on Regional Economic Studies Institute (RESI)’s recently launched “Knowledge Center” page.


Our first tutorial focuses on how to write Standard Operating Procedures (SOPs), which are important documents for any business (see my previous post titled Three Scary Words). SOPs can provide your organization with the ability to understand the way your company works, streamline your communications processes, and increase your efficiency. This tutorial will explain the 5 Steps to Creating SOPs. 




In my previous blog CAIRE and Race to the Top, I provided an overview of what CAIRE means, who is involved with it, and how it relates to Race to the Top (RTTT). CAIRE‘s main function is to independently evaluate 54 distinct but interrelated projects. Many of these projects will be used to provide guidance and direction as the state endeavors to successfully implement the new Maryland Common Core Curriculum in all Maryland schools.


The past year and half, CAIRE has been evaluating the implementation of RTTT in Maryland. Evaluating a large program presents itself with many challenges as to how and what to evaluate. The goal is to provide a thorough and objective evaluation that adds value for the program to make course corrections. To break the evaluations into manageable pieces, the Maryland RTTT program was conceived with three phases:

  1. Process/Product
  2. Utilization
  3. Impact

caireRepresentatives of the CAIRE Team met each week with different RTTT project managers to discuss the process/product of the 54 RTTT projects. The purpose of these meetings was to review and document the status of each project from a management perspective, identify any risks associated with its completion, and if warranted, make recommendations to RTTT leaders to improve the project’s management. An evaluation plan was created that captures specific project information and then asked questions relating to the management of the project. The challenge in this phase was coordinating with 54 different project managers in arranging a time to meet. The end product was a report on the 54 projects with associated recommendations.


The second phase of RTTT was to measure the utilization of each of the projects. This phase, which is where CAIRE is currently, focuses on measuring the utilization of the product/service for each project being developed. One of the challenges that is being overcome in this phase is how to measure utilization given some projects would not be completed until near the end of the grant. For those projects that were completed, the third phase could be measured at this point in time.

  1. The CAIRE Team devised an evaluation plan that defined the product or service being produced and set out to discuss with each of the project managers the outcome of their project.
  2. With that information, an implementation plan was developed to measure how its stakeholders and end users were utilizing the product or service.
  3. After each individual project has been evaluated, the CAIRE Team will analyze the results and provide a final report with recommendations.

The challenge in this phase was again the schedule. Not only is the CAIRE Team meeting with the 54 PMs, but simultaneously meeting with stakeholders and end users to measure utilization. It’s a tight schedule with very little room for adjustments, but the CAIRE Team is meeting the challenge.


The final phase, impact, is the most challenging in that to measure impact, you must first understand what constitutes impact. This phase is on the horizon and discussions about how to measure impact for the RTTT in Maryland are underway. To stay updated on the progress of RTTT, please visit the MSDE web site.


Within the RESI-Business Analysis and Management (BAM) group is a team of business analysts, project managers and economists working on the CAIRE project. CAIRE stands for the Center for Application and Innovation Research in Education and the team is getting ready to move onto to Phase II of one of the most important projects in the State of Maryland.


This post will be the first in a series about BAM’s work with CAIRE.  Throughout the series, I will catch you up on the newest developments and impacts the project is having on improving education in Maryland.  I’ll inform you about this exciting project that crosses universities, Local Educational Agencies (LEAs), state, and federal government agencies.


First, what is CAIRE?

CAIRE is a center of the University System of Maryland housed at Towson University (TU).  It’s supported by faculty and staff from several Maryland universities including, BAM staff members and faculty from TU’s College of Education, University of Maryland Baltimore County, Loyola, and University of Maryland Eastern Shore.  Our team, headed by TU College of Education Dean Dr. Ray Lorion, was created to assist the Maryland State Department of Education (MSDE) with evaluating the activities associated with the Race to the Top (RTTT) grant.


What is the RTTT grant?

The RTTT grant is a federal grant from the US Department of Education (USDE) that MSDE applied for and received.  The RTTT grants funds are aimed at boosting student achievement, reducing gaps in achievement among student subgroups, turning around struggling schools, and improving the teaching profession.  The RTTT grant funds are being used to support 54 projects over a four (4) year period (2010-2014) in meeting these objectives.



What is the purpose of  these 54 projects?

All 54 projects were developed to meet the goals set in the RTTT program and are wide ranging.  These projects are all centered on moving Maryland’s educational system to the Common Core State Standard.  Some projects involve tracking student performances as to better fulfill individual students’ educational needs while others center on ways to improve teacher retention and training and yet there are others that are looking at new and innovative ways to incorporate STEM (science, technology, engineering, and mathematics) into all aspects of the curriculum.  Maryland has one of the best school systems in the country and the RTTT grant is assisting Maryland in solidifying this for years to come.


So what exactly is CAIRE (with BAM’s help) doing?

CAIRE‘s main function is to independently evaluate these projects by looking at process/product, utilization, and the impact the 54 projects are having on the education of Maryland’s students.  The goal of CAIRE’s efforts is to assist MSDE with implementing the Common Core State Standards for Maryland.


I can’t wait to share how our evaluations progress and the innovative ways we are developing to quantify this information.  Sounds like 2013 will be busy for the CAIRE team!


The Business Analysis & Management  group within the Regional Economic Studies Institute (RESI), recently had the opportunity to work with members of the Department of Information Technology or DoIT.  DoIT is responsible for overseeing the State’s major IT projects.  As the State moves forward on more and more technical projects, the need for DoIT to have streamlined and easily understandable documentation has increased.  Our work focused on using our group’s biggest skill sets: technical writing and process analysis. Working in collaboration with DoIT staff members, we:

  • Organized and clarified important information and processes used by DoIT into easily readable documentation.
  • Documented best practices for agencies to use when writing Request for Proposals such as how to write a good quality “Minimum Qualifications” and “Requirements” sections.
  • Wrote Standard Operating Procedures for the oversight of major information technology development projects. It is anticipated that this information will eventually be available on line.

There is an art to taking information from various sources and making it into a coherent, readable, understandable and USABLE document.  What has always set our group apart is that we not only understand how an agency works, but we create lasting relationships in the process.  Being able to take documentation that already exists and re-define, fine tune, and streamline it takes a unique skill set. But being able to develop working relationships with the client and in turn have them provide the necessary information and want to work with us to create the documentation is where we excel.

While our team will be moving on soon to other projects and places, I know we’ll all take with us a stronger knowledge-base of how DoIT works and an appreciation for all the work they are doing every day to support the citizens of Maryland.