Sharks in the Moat

by Phil Martin

  Retirement

  Decommission (delete) or replace software

  Dispose data to avoid data remanence

  Figure 164: Software Supply Chain Risk Management Processes

  Finally, at some time in the future the product will need to be retired, and this phase covers all activities associated with disposal to prevent information disclosure.

  The remaining topics in this section are grouped according to the 8 steps of the SCM acquisition lifecycle.

  Chapter 50: Step 1 - Planning

  The relationship between acquirer and supplier can be one of three different forms as shown in Figure 165 – subcontracting, staff augmentation, or licensing.

  Figure 165: Types of Relationships Between Acquirer and Supplier

  Subcontracting is a type of ‘work-for-hire’ relationship and puts the burden of software development onto an external party. The subcontractor creates the software for the acquirer, who owns the result of all work. This work is normally done at the supplier’s facility, and management of people is normally carried out by the supplier.

  Staff augmentation is another type of ‘work-for-hire’ relationship, but with this approach the acquirer will usually directly manage the subcontracting individuals, and work is often performed in the acquirer’s own facilities. Employees from both the acquirer and supplier work in close collaboration.

  When we enter into a licensing agreement, we obtain completed software from an external party to be used as-is, or with modest customizations. This is sometimes called an “arm’s length licensing” relationship.

  Each supplier within a supply chain might also enter into their own agreement with another supplier in one of the three relationship types. In general, supplier risk management will begin with finding the suppliers, followed by the protection of intellectual property.

  Chapter 51: Step 2 - Contracting

  The proper selection of a supplier is crucial to success and settling on an inadequate party is not something you ever want to do. The process to select the right partner can be grueling but should be considered carefully.

  A company I once worked for decided to outsource work to India and selected a vendor solely because there was already an existing relationship and the documents could be signed in a quick fashion. Myriad warning signs were ignored, such as a propensity to miss milestones, the suspicious nature of remote contractors disappearing for weeks at a time, and the inability to perform proper code reviews. One year later after untold millions of wasted dollars, the decision was finally made to move the work to nearshore. While nearshore prices are essentially double as Asian-based offshore resources, the quality and volume of work tripled. If only someone had taken the time to perform due diligence up-front, a lot of frustration and wasted money could have been saved.

  When evaluating a potential supplier, we need to take into account three areas – the past and current organizational track record, how mature their internal processes are, and the people they will be using for the project. This is called a pre-qualification assessment.

  Assessing the Organization

  The financial stability of a supplier is critical to ensuring they will be able to allocate sufficient resources when addressing defects and vulnerabilities. Mergers, lawsuits, losses and sell-offs are all events which can completely derail a supplier’s ability to focus on the project. While it is difficult to predict the future, there are things to look for. As an example, when choosing nearshore vendors, I once took into account that a specific supplier had diversified into multiple markets and therefore would be able to handle a market downturn better than competitors who were dependent on a single market for their income.

  A supplier’s organizational focus should be ascertained, as this can help to telegraph future actions. For example, if a vendor’s primary focus is on manufacturing hardware components, and provides staff augmentation on the side, it may be a greater risk to go with such a supplier as they may decide to divest non-core businesses on a whim. If using a foreign supplier, the relationship that the country of origin has with your own country must be considered. A U.S. company using a Russian supplier must be careful. Consider Kaspersky Lab, the world’s leading expert on malware detection and removal – it may or may not have ties to the Russian government which has been antagonistic to the United States for over 70 years. Although there has never been definitive proof of collusion between Kaspersky and the Russian government, it does cause an elevated level of risk, particularly if a company is performing work on behalf of the U.S. government. However, we need to be careful to not overstate the influence of national interests on a supplier – the risk due to the absence of proper security controls is a FAR greater concern. In short, where services are performed is not as important as how the services are performed.

  A supplier must be in compliance with both its own internal policies as well as external regulatory or privacy policies. After all, if a supplier does not do a good job of staying in compliance with their own internal processes, how important do you think it will be for them to stay compliant with our policies? Additionally, the supplier should demonstrate a high-level of familiarity with any policies we expect them to align with. If we deal in PHI, and a supplier doesn’t already understand HIPAA regulations, we should probably move on and consider other candidates.

  Figure 166: Requirements and SLAs

  When dealing with suppliers, the SLA reigns king in terms of our ability to control and enforce non-compliance. In short, if it isn’t written into the SLA, then it “ain’t gonna happen”. SLAs can contain both incentives and penalties, but a review of a supplier’s potential SLA is very revealing in how they view contractual obligations. SLAs can be of two types – requirements-dependent and requirements-based, illustrated in Figure 166. A requirements-based SLA includes the requirements that a supplier must meet within the SLA. A requirements-dependent SLA assumes that requirements are external to the SLA, and therefore it will be difficult to define service levels within an SLA.

  SLA Metric Category

  Coverage

  Performance

  Reliability in the functionality of the software. Is the software doing what it is supposed to do?

  Disaster Recovery and Business Continuity

  Speed of recovery of the software to a working state so that business disruptions are addressed.

  Issues Management

  The number of security issues that have been addressed or deferred for future releases. How many bugs have been fixed? How many remain? How many are to be moved to the next version?

  Incident Response

  Promptness in responding to security incidents. This is dependent on risk factors such as discoverability, reproducibility, elevated privileges, numbers of users affected, and damage potential.

  Vulnerability Management

  (Patch and release cycle)

  Frequency of patches that will be released and applied, and the measurement as to whether the process is followed as expected.

  Figure 167: SLA Metric Categories and Coverage

  Data classification can help in determining requirements for an SLA. When classifying data, we can arrive at quantitative values such as the maximum tolerable downtime (MTD) and the recovery time objective (RTO). These can be directly referenced in an SLA, such as “the system will not encounter down-times in excess of 4 hours”. The total cost of ownership, or TCO, represents the real cost of owning a system, and can be directly affected by how well an SLA is written regarding CIA. A key performance indicator, or KPI, is an important metric to include in any SLA as it allows us to evaluate how well a supply chain is progressing. SLA incentives are often based on achieving specific KPIs by a set date as a carrot to motivate suppliers. Figure 167 shows some common SLA metrics and what coverage each provides.

  A supplier’s past performance with other customers should be examined to assess how the supplier will behave after the product has been delivered. This can be accomplished by requesting a list of
previous customers, or by using a third-party to provide an assessment.

  Assessing Processes

  A supplier’s software engineering process should be examined to ensure they have a structured process – this is required if we hope to incorporate any level of security into the overall delivery process. By gaining an understanding of the supplier’s lifecycle and how security is implemented in each phase, we will be able to predict how secure our own product will be. Some questions to ask the potential supplier are the following:

  How is your software development process structured?

  What artifacts are generated along the way?

  Will you be outsourcing software development, and if so what checks and balances will you be implementing to ensure security?

  Do you have a threat modeling process, and is there a threat model for the software you are designing for us?

  What kind of design, architecture, code and security reviews do you conduct?

  How is the software tested against functional and security requirements?

  How do you implement access control to the source code base?

  Has the software been tested and verified as secure by a third party?

  How current and accurate is the documentation that comes with the software?

  The supplier’s processes around vulnerability and patch management must also be examined. Any viable supplier should exhibit the resources and expertise needed to release patches in a timely and secure manner to address vulnerabilities discovered in their software. Evidence of this capability includes the following signs:

  They routinely collect input on vulnerabilities from databases such as the OWASP Top 10, NVDB or OSVDB.

  They routinely analyze the applicability of top vulnerabilities against their own software.

  They can articulate discovered vulnerabilities using common terminologies all while referencing relevant specifications such as CWE or CVSS.

  They provide remediation steps to address discovered vulnerabilities within acceptable timeframes.

  They proactively provide points of contact and escalation plans to address vulnerabilities as they are discovered.

  They have a proven track record of responding to vulnerabilities in a timely fashion.

  Regarding the last bullet point of having a good track record of being responsive, the alternative will be a supplier who is unable or unwilling to patch vulnerabilities before they are exploited by an attacker. Avoid this supplier at all costs!

  Assessing People

  The individuals within a supplier’s organization must be properly vetted to see if they are familiar with current and emerging security threats. In fact, you should explicitly state the knowledge areas and competencies you expect for supplier employees to possess. If there is a lack of integrated security processes in the software development lifecycle (SDLC), then you can be assured that the individuals do not have the necessary security training. Anyone with privileged access to code and data should be screened for a criminal history, especially those with felony charges involving computer crime. Find out if employees undergo training in the latest security threats, and how often such training is carried out. The supplier should also be able to show an acceptable level of background check and screening processes that are routinely executed.

  Response Evaluation

  Another method we can use to evaluate how well versed the supplier’s people are with security is to evaluate the response to our inquiry. How well the response addresses security concerns without us having to pull that information says volumes about both their previous experience as well as how important security is to that organization.

  The usual way that an acquirer will advertise their interest in obtaining technology is to use a request for proposal, or RFP, a request for information, or RFI, or perhaps a request for quote, or RFQ. By far the most common method is to use an RFP. It is critical that the RFP contains security requirements, as it becomes much less likely that proper security mechanisms will be delivered in the final product if the RFP is lacking in this area.

  The first step in generating an RFP is to create a work statement. Within this document, it is important to state what trustworthy software looks like along with specific security requirements, resulting in an assurance case. An assurance plan makes sure that we develop and maintain the assurance case properly. When stating security requirements, quantitative measurement criteria should be included as much as possible, as this prevents ‘guessing’ on the part of the suppler. The amount of time a supplier has to respond to the RFP must be clearly stated in terms of absolute dates. If late submissions are allowed, the terms must be spelled out to prevent misunderstanding. This helps to reduce scope creep later.

  The evaluation criteria used to assess responses must be predefined before responses are received, as they should be included in the RFP itself. Some common examples of evaluation criteria are the following:

  How well the supplier understands the requirements.

  The proposed solution.

  The experience of personnel that will be involved on the project.

  Valid references from past projects.

  The amount of required resources, cost and schedule.

  How intellectual property ownership and responsibilities will be handled.

  To ensure a fair and uniform evaluation, the same criteria must be used for all responses. The evaluation team should include membership from the various parties involved and stakeholders, such as development, infrastructure, operations, legal, security, product, etc. This ensures that all aspects of the project have a say in both the RFP and selection of the final supplier.

  Contractual Controls

  Ideally, an acquirer should never start writing software with a supplier until the contact has been finalized. In the real world, if there is a strong prior relationship and trust, this is often bypassed in the interest of expediency while legal works out the details. That doesn’t mean it is the right thing to do, it just means that is what often happens. I personally have taken this tactic with great success on multiple occasions when I enjoyed a preexisting healthy relationship with a vendor. But it is not without risk and the individual making the decision to expedite work without legal contracts in-place must be willing to accept the consequences if the relationship does not survive the legal stage.

  Beyond stating the expectations and responsibilities of both parties, a contract must also address conditions and consequences of contract breaches. If included in the contract, then the associated terms will most likely hold up in a court of law. Obviously, we always want to trust our partners and enjoy a great working relationship. But take note of one simple fact – behind every great working business relationship you will find a contract with sharp teeth. A business relationship is not a marriage where you must be willing to put the other person first – a business relationship will always involve two parties looking out for themselves first and foremost. Sometimes we hear great stories about how one company bent over backwards to help out a partner in time of need, but in these cases, one of two realities were at-play:

  1) The bosses of each company were great friends, probably golf buddies.

  2) The ‘helpful’ company was simply taking a longer-term approach to the relationship and was gambling that in the future it would benefit with a near-term sacrifice.

  In either case a strong contract will protect the company. Contractual language should, at a minimum, contain the following:

  References to the applicable regulations and standards that both companies will need to remain compliant with.

  The software development methodology (SDLC).

  Personnel qualifications and training required to assure security trustworthiness.

  Specific security controls that must be implemented.

  How integrity of the development environment is to be maintained.

  How integrity of the distribution channel will be maintained.

  A right to conduct security code r
eviews within a given timeframe after receipt of the software from the supplier. This includes the methodology used to address issues that are found.

  Testing terms used to verify and validate security controls, including the use of both self-testing and third-party testing. Third-party testing should reference penetration testing.

  Who will own the code and intellectual property developed during the engagement.

  Well-defined acceptance criteria.

  The certification and accreditation process the software will need to pass through.

  An expectation that the supplier will properly address errors and vulnerabilities within a specific time period.

  The patch and vulnerability management process that the supplier will need to align with.

  Any warranties regarding malicious code discovered after delivery.

  Any software or service reliability guarantees.

  A requirement that certifications of originality will be provided to ensure the supplier does not encroach on works owned by other entities unless licensed.

  When laying out required security controls, both coding and configuration requirements must be included. Some more common examples that should be referenced are the following:

  Input validation

  Secure libraries and frameworks

  Safe APIs

  Authentication checks

  Authorization checks

  Session management

  Error and exception handling

  Logging