Introducing
The Testability Director Version 3.2

Common Criteria for Selecting your System Integrator
 (and Five Others you probably did not think of)

 By:

How to Select the Right System Integrator for your Next Project

 By: Don Holley, VI Technology

Also View/Participate in the Survey

System Integrators (SI’s) of automated test systems show value by collaborating with clients providing the technical resources, know-how and internal processes required to take a test need into a successfully completed test solution. A SI should essentially be an extension of the client’s organization.

But, no matter how capable or skilled a SI may be in the development, manufacturing and verification of automated test systems, there is no value-added until the SI truly understands the client’s technical and business requirements and expectations.

Correspondingly, the client must understand his or her own integrated test system needs and timelines, before making a choice of the right SI.

Clients with past experience using SI’s know the questions to ask and how to proceed when doing complex projects with SI’s. First time clients, on the other hand, will not find it quite as simple and may make some poor assumptions. There are a number of important criteria to consider when evaluating and deciding upon a SI. Some of the criteria are obvious; some not. Outlined here are key criteria to use when evaluating SI’s….and then some.  Beyond the typical and obvious areas for evaluation, we outline five additional criteria that most selectors of SI’s typically do not think of.

§         Choose a SI who really understands your test requirements.
Any SI must have the skilled people, knowledge and past experiences that map directly to understanding and solving the client’s unique test problems.

To prove understanding of test requirements, (their “technological and professional core competencies”), client’s should ask the SI to show examples of past work that closely aligns with their specific requirements. Ask the SI to supply a number of client references and check them out.

§         Verify the SI has the skilled resources and processes needed to successfully complete your project on schedule
Visit the SI. Meet their engineering, quality, manufacturing and operations teams. Review the tools and processes they have designed and implemented on past projects (hardware design, software design, purchasing, manufacturing/operations) to confirm how “proven” they are. Tour their facility to confirm the SI in fact has the resources and processes to successfully complete any project on time

§         What process does the SI use for accurately scoping, sizing and pricing test projects?
To ensure success, make sure the SI has the skilled people and processes to accurately size/scope/price/schedule the project, before starting the project. Does the SI have the System Engineering “breadth of experience” to ask “all the right questions”? …How to make sure the system Acceptance Test Plan is accurately defined, as well as things like user ergonomics, networking, data storage/analysis, training, support, and other non-technical requirements? Ask the SI to share their Work Breakdown Structure (WBS). Talk to the people responsible for these steps and review actual examples where this process has been successfully used in the past.

§         Define the kind of business relationship you want to have with an SI….Make sure the SI is the “right size” to meet your needs
Large, complex projects require a SI to have scale, a full breadth of resources, creativity and innovation skills, along with solid past experience. On the other hand, a smaller SI may handle smaller projects more efficiently.

In some cases, you only need a SI to address a single specific test requirement. In other cases, you may be looking to establish a longer term, strategic technical and business alliance with an SI. Either way, clearly define this before starting an evaluation of SI’s, (besides defining goals, budgets and project schedule needs). How important this is may also play a role in where the SI needs to be located.

Finally, is there a requirement for security of Intellectual Property? Does the SI meet any security clearance requirements? Visit the SI to see if they are sized right, and facilitized/cleared for secret project-related work (if required) to meet project needs.

§         What  kinds of extended post-sales support services does the SI offer?
Confirm the breadth and depth of support service resources. What kind of warranty support is offered? Are obsolesance management services available? Besides domestic support, does the SI also provide international/worldwide services?

And the Five Things you probably would not think of….

1. Confirm the level of relationships the SI has with industry OEM’s and other industry experts  
It’s critically important that any SI be fluent in leading-edge test technology solutions and is an “independent” SI services provider. When operating independently, the client is assured that the SI is acting more as a “consultant” in terms of proposing alternative solutions to address test problems vs. an SI who may be connected or has a “bias” towards certain OEM’s products and services. Clients must confirm that the SI is capable of proposing the most cost effective solution that meets their technical, business and scheduling requirements. Likewise, if the client already has a list of materials of certain brands to be integrated into a complete test system, they should make sure the SI has prior applications and integration experience with those brands/OEMs. Finally, no SI has all the answers and therefore may have to outsource. Make sure the SI has relationships with other industry experts.

2. Choose an SI who really understands your test requirements/problems……..and is willing to provide you with any system IP, source code, drawings and/or manuals
In the end, it’s the client’s system, not the SI’s. Therefore, the client should receive any system-related IP and the SI should be happy to provide it. Make sure this is understood upfront, before the contracts are signed and project work is underway. The SI should be willing to provide whatever is needed to NEVER have to use that SI again, if the client so chooses.

3. Choose an SI who really understands your test requirements/problems……..and has the financials to get the job done
Credit check to verify any SI’s financial strength and stability! Also confirm they have the purchasing power (volumes) to command larger discounts from OEM’s needed to complete any project. Also, verify how the SI provides quotations/pricing. It should be done on a firm, fixed-price basis.

4. Review examples of the SI’s Documentation
Upon delivery of a test system, the documentation will be the client’s primary point of reference, so it better be right! Check for thoroughness and quality. Ensure the SI’s Project Status Reports are provided regularly and are easy to understand. Confirm the System Operation and Maintenance instructions/procedures are complete and easy to understand.

5. Verify that the SI has the certifications and quality standards that any project requires operationally in-place….today
During site visits, review the SI’s quality processes and procedures. Confirm claims of certification and compliancy with industry quality standards. Meet and interview their Quality Control Manager. Ask him to give you a tour of the facility to show their quality processes and procedures in use. Also ask about the results of any audits or certifications that the SI’s customers have done as well.

Don Holley, VI Technology

Depending on your specific needs, system integrators can deliver a wide range of services including software development. This can encompass custom turnkey solutions such as commissioning, training and on-site support or consulting where special technology or industry domain expertise is required. System integrators (SI) add-value to your organization by augmenting your own design, engineering and project management resources so that you can focus on your core competencies. An SI also gives you the flexibility to increase staffing in order to meet tight project schedules without reshuffling internal resources and priorities.

Understanding your project and staffing requirements and developing a comprehensive requirements document will help facilitate your search and selection of a qualified candidate. A list of typical questions to consider when evaluating and selecting a system integrator are listed below:

• Has a scope of work been developed detailing project requirements?

You may have a good handle on your project scope, but if not, you may need to first develop a requirements and scope of work document. You may also want to consider contracting a system integrator to provide this deliverable. Many times the SI has the domain expertise and can perform an onsite study and assessment necessary to create a scope of work (SOW) document. You can then send this SOW out to your list of SI candidates to solicit proposals.

• Who will be responsible for managing the project?

The system integrator will typically be the project manager if you are going to turn the entire project over to them with expectations that they are going to deliver and commission a turnkey solution. You will be responsible for overall project management if you are just subcontract parts of the work out to the system integrator? Again, this needs to be determined upfront.

• Does the project require travel outside of the geography served by the integrator?

Depending on resources and size, a system integrator’s services are sometimes most cost effective within a specific geographical area. This should be a consideration if your project will require the system integrator to travel in order to provide onsite installation and startup support.

• What depth can the SI offer?

Make sure that SI you select has sufficient resources necessary to complete your project within the specified time and budget. SIs range from one-man consultants who work out of their house to “large” SIs that employ 100’s of people located in multiple geographies.  A one-man consultant, although usually lower in price/hour is often more expensive in total cost because of the efficiency gains a larger SI can provide, especially on larger projects. In addition, smaller SIs will not have the depth of employees to call upon should there be unforeseen personnel or project issues.

• Does the integrator have experience working with the customer’s preferred equipment suppliers?

Many times, certain system integrators also have experience working with certain suppliers such as test, measurement and automation equipment for example. This can benefit your project because the SI will not have a learning curve and can be more efficient developing a solution using hardware and software from a supplier they have experience with. You should also see if the SI has personnel who have achieved certification from the supplier ensuring they are qualified to work with their products. Examples of supplier certification programs include National Instruments Certified Alliance Partner, Microsoft Certified Partner and the Control System Integrator Association Certified Registered Member programs. Over half the engineers at our company have achieved one or more supplier certifications.

• Is custom software and hardware development required or can off-the-shelf products be used?

Over time, an SI works on many projects and ends up solving the same or similar problem multiple times. For example my company, VI Technology, worked on many test system projects and saw a recurring need for a test data management solution for collecting, managing and accessing test data generated by automated test systems. We developed an off-the-shelf product, Arendar Enterprise Test Data Management software, to address this common test system integration problem. Our customers can use Arendar as an off-the-shelf test data management solution or we can use Arendar in conjunction with our test system integration services to deliver a turnkey enterprise-wide test data management solution at a lower cost than developing custom software.

This list of criteria is not comprehensive. You can add additional criteria to this list based on your project needs when selecting a system integrator for your next project.

Once you have identified your project requirements, budget, schedule and selection criteria, what are the steps involved in selecting a system integrator?

1. Develop a short list of SI candidates
2. Meet with representatives from the SI to discuss your project
3. Receive a proposal from the SI
4. Decide which SI to outsource your project

One good source of information for selecting and working with SIs is provided by the Control & Information Systems Integrators Associations (CSIA). You can visit their web site and download a free copy of their guidelines, "How to Work with and Select a Control System Integrator.”

Of course the evaluation and selection process can be shortened if you have prior experience working with an SI with the necessary skills required for your particular project. Having a good working relationship with an SI complements your existing resources by giving you the flexibility to outsource projects and flexibility to augment your staff if necessary.

• If your friend forwarded this newsletter to you, please register as a member and receive The BestTest Newsletter -- absolutely free!
• If you wish to update your news preferences or cancel the subscription, please unsubscribe.
• If you have any questions, please email experts@BestTest.com

Get the widest selection of test related books and software at the BestTest Online Store.

Upcoming Events: