A survey on common criteria (CC) evaluating schemes for security assessment of IT products

Paper organization

The rest of the research is organized in seven different sections which discuss all major contributions of paper. ‘Related Work’ presents a summary of relevant literature. ‘Research Methodology’ presents the research methodology employed for carrying out the survey. ‘Survey of Country Level Implementation of CC Schemes’ discusses in detail the country level implementation of CC schemes of four developed as well as CC-certificate authorizing states; US, UK, Singapore and Netherlands. ‘Comparative Analysis’ presents a comparative analysis of the studied four CC schemes. ‘Best Practices to be Followed while Establishing a New Country Scheme’ proposes best practices to be followed while establishing a new country scheme. Lastly ‘Conclusion’ concludes the paper and ‘Open Challenges’ presents open research areas.

United States scheme

Introduction: CCEVS or the NIAP is a scheme responsible for the security evaluation of COTS products in accordance with the CC. The validation body under the CCEVS is led by a Director and Deputy Director selected by NSA and NIST personnel (NIAP, 2019a). NIAP performs security assessments against the NIAP-approved PPs. The scheme consists of five important entities including laboratory director, approved signatories, authorized representative, evaluation team leaders, and senior evaluators.

Accreditation procedure of the testing facility: Organizations interested in performing evaluations and acquiring CCTL go through a series of steps. In the US, NVLAP is accountable for accrediting the testing facility and Table 2 presents its tabular form. Moreover, following is the list of NIAP and NVLAP requirements for acquiring CCTL approval.

1. NIAP requirements: NIAP verifies the specified requirements (NIAP-Certificate, 2017) by confirming the ’letter of conduct’ being submitted by the applicant CCTL. If all requirements are fulfilled, then the status of NIAP-approved CCTL will be granted and documented.

2. NVLAP accreditation: It entails a candidate CCTL to show conformance with the methodological and technical criteria to conduct security evaluations of IT products (NIAP-Certificate, 2017). The assessment process consists of proficiency testing, an initial on-site visit and laboratory management system review. Important steps of the accreditation process are management system review (NIAP-Certificate, 2017), technical requirements for accreditation (NIAP-Certificate, 2017), proficiency testing, and NVLAP review.

The flow of evaluation and certification of IT product: The validation and evaluation of COTS along with the responsibilities of sponsors, CCTL, and the scheme in terms of evaluating IT products are discussed below. Once the sponsor has developed the ST and the strategy to share technical details, evaluation should be started (NIAP, 2019b, Gossamer-Laboratories, 2019; NIAP, 2014).

1. Phase I: First and the foremost step is to choose an appropriate CCTL. Another important step is to ensure that conflict of interest should not be present between evaluators and consultors (Gossamer-Laboratories, 2019).

2. Phase II: This phase begins with the check-in meeting with the sponsor and concludes with getting a final verdict (NIAP, 2019b). The responsibilities of evaluation facility with respect to NIAP are shown in Table 3.

3. Phase III: This phase comprises delivery of the evaluation document, PCL posting, and certificate issuance. In this phase, CCTL will provide ETR, ECR, VR, and TRRT to the validator. Once the review has been completed, the validator will complete VR. The NIAP director, in turn, will either prepare a CC certificate and forward it for signature or notify the sponsor and CCTL of the unsuccessful evaluation along with the rationale. In case of acceptance, mutually recognized CC partners will be notified (Gossamer-Laboratories, 2019; NIAP-Certificate, 2017).

4. Phase IV: Once the certificate has been issued, it is essential for the scheme to perform certificate monitoring and assurance continuity, which are discussed in the subsequent sub-subsections (Costa, 2019).

Validation process: It is the process of independent assurance that the evaluation has been conducted in accordance to NIAP policies and CCTL findings drafted in the ETR. CCTL will expedite the overall evaluation process by eradicating the requirement for the conventional VORs (NIAP, 2014; Gossamer-Laboratories, 2019; Jacobs, 2015; Leaman, 2015; NIAP-Certificate, 2017).

Surveillance: NIAP performs surveillance tasks in the following forms.

• CB validator: This process provides supervision at different milestones during evaluation. Important milestones include KO, IVOR, FVOR, TVOR, and evaluation conclusions (Jacobs, 2015; Leaman, 2015). Based on NIAP, Table 4 presents important responsibilities of CB.

• Scheme and accreditation body: It is essential for the validator to notify scheme management about the deficiencies on the part of CCTL. The roles and responsibilities performed by the NIAP is presented in Table 5.

Communication management: It can be done in the following ways:

• Role of validator in managing communication: The role of validator is extremely important in managing communication between the evaluation facility, CB, and scheme (Jacobs, 2015).

• Validation of evaluation results: Its role is significant in verifying and communicating evaluation results, records, and ETRs in terms of completeness and technical accuracy.

• NIAP representative: Another important responsibility of validator is to serve as a NIAP representative by providing a central point of contact between CCTL and NIAP. Moreover, another important role of the validator is to forward technical queries to suitable TRRT for comment and review.

• CCTL support: Validator provides support to CCTL by managing communication with the evaluation team when required and by ensuring that the team is aware of all suitable test methods and evaluation techniques. Moreover, it suggests information to be incorporated in records and ETRs for effective and efficient evaluation.

Suspension or withdrawal of accreditation/approval: If CCTL has not been organized with NVLAP and NIAP requirements then its status will be suspended or withdrawn. If CCTL status is ’withdrawn’ then CCTL will cease all evaluation activities and the lab has to reapply for accreditation (Jacobs, 2015).

Records management: The major activities involved are records and procedures, orientation meeting for validators to get an idea about records’ availability, storage, processing, and management. Moreover, to comply with the quality system of a scheme, it is imperative for the validators to keep the record of the work in an organized manner. To serve this purpose, records should contain a unique identifier at the top right in a standardized format. Furthermore, it is significant for validators to identify and protect the specified proprietary information. Official records should be closed out too and forwarded to the records manager within 30 days of final package delivery. In addition to that, the scheme will keep the evaluation record for at least 5 years.

Assurance continuity: Assurance continuity describes a method to minimize repetition in security evaluations. Maintenance is about the procedure, the developer practices to update documentation of a changed TOE whereas re-evaluation process assesses changed TOE (Jacobs, 2015). Due to the change, the CCTL/developer should submit an IAR for the products at least 30 days before the assurance maintenance. In case of major changes, CCTL must conduct tests and generate ETR. Finally, new VR, PCL, and certificate will be issued by CB. With respect to NIAP, Table 4 presents important responsibilities of CB. In case of minor changes, the developer fixes the bugs and the addendum will be made to the PCL along with documented fixes and patches.

Dispute handling: It is the responsibility of validator to respond to ECR inquiries and TRRT in timely fashion and scheme should support them (Gossamer-Laboratories, 2019).

Interpretations: To perform TOE evaluation, evaluation applicable in accordance with NIAP policy, CEM, and CC interpretations should be applied. The CCTL is accountable for the identification and application of appropriate interpretations.

Singapore scheme

Introduction: SCCS is managed and owned by CB under the scope of CSA. The security evaluation of IT products is conducted by the approved CCTL, which is compliant with SCCS and accredited by SAC followed by the verification of results by the CB. The management board of SCCS is the ultimate authority dealing with the evaluations being conducted and certificates being issued (SCCS, 2018b).

Accreditation procedure of the testing facility: Evaluation facility should be accredited by SAC or by other recognized body in compliance with ISO-17025. The lab must have a security policy compliant with ISO 27001 (ISO, 2013b) and ISO 27002 (ISO, 2013a). To gain accreditation, CCTL must be capable of performing evaluations at EAL-4 and be compliant with CEM and SCCS policies. Table 2 presents the procedures adopted by SCCS to accredit evaluation facility.

The flow of evaluation and certification: The evaluation will be conducted in the following phases (SCCS, 2018c).

• Pre-evaluation phase: The pre-evaluation phase involves feasibility study which states that if the sponsor wants to acquire a certification of an IT product, he will involve CCTL to perform the evaluation under legal contract by claiming conformance to a scheme-approved national PP, endorsed cPP or certified PP recognized by CSA. The products not claiming conformance with any of the above-mentioned conditions will be accepted up to EAL2+ (SCCS, 2018b). However, for certifying higher levels, the explicit written requirement by Singapore Government Agency is essential. Based on the evidence provided by sponsor, CCTL will conduct a feasibility study to analyze the cost, time, and scope of the evaluation. The major responsibilities of laboratory facility with respect to SCCS are discussed in Table 3.

  Application review is also included in this phase which states that once the application has been received, the CB will review the form, ST compliance with CC and updated EWP. If CB decides to conduct the evaluation, a ‘Letter of Acceptance’ will be provided to the sponsor and in turn, the confirmation will be granted to CCTL. A certifier will be assigned to supervise the overall evaluation (SCCS, 2018c). Based on the Singapore scheme, Table 4 presents important responsibilities of CB. After accepting the TOE for evaluation, CB will call kick-off meeting with the evaluation working group to confirm and discuss EWP to comprehend evaluation scope and address the applicability of the project’s supporting documents. TKM is considered successful if all entities (CB, CCTL, sponsor) approve TKM meetings (SCCS, 2018c).

• Evaluation phase: Upon successful closure of TKM, CB will list particular IT product under evaluation and will perform assessment in accordance with the EWP. Test plan for AVA and ATE should be provided to CB at least 2 weeks before EPM. At least, one EPM must be conducted after the completion of ASE (Common-Criteria, 2017) and one before and after the commencement of AVA and ATE (Common-Criteria, 2017). Table 4 presents important responsibilities of CB in accordance with SCCS. Moreover, CCTL must perform evaluation activities in accordance with ISO 17025 (SCCS, 2018c; SCCS, 2018a).

  SER and ETR are the technical reports which are involved in evaluation phase. SER must be submitted to CCTL and CB upon completion of each evaluation task. SER contains verdicts of assessment activities from CCTL along with their justifications. The CB, in turn, reviews the SER and updates observations in SER-RR. For independent penetration and functional tests to be performed on TOE, CCTL should document test requirements, configurations, details of exploitable vulnerabilities, and scripts in SER. The major responsibilities of laboratory facility by considering SCCS are discussed in Table 3. ETR includes the information derived from ORs and SERs, which are then submitted to the CB. Vulnerability analysis must be conducted within 6 months prior to ETR submission otherwise CCTL should perform fresh vulnerability analysis. For international recognition, CCTL should be accredited to ISO-17025 before final ETR submission to the CB (SCCS, 2018c; SCCS, 2018a).

• Conclusion phase: Finally, Task Close-down Meeting will be arranged once the CB approves the ETR. During EPM, the CCTL will provide a brief synopsis of all evaluation tasks. The approved ETR will provide a baseline for preparing CR, which contains the final verdict regarding successful/unsuccessful evaluation.

• Certificate awarding phase: Once the evaluation has been completed, the process of awarding certificate begins. The CB will prepare a report containing forms, ST, ETR, CR, and CC certificate. Upon approval by CB’s head, the certificate with developer name will be available on CPL for international recognition. This certificate will be valid for 5 years, which can be extended through assurance continuity (SCCS, 2018a).

Surveillance: The identification of roles and responsibilities taken into account by the SCCS are presented in Table 5. This scheme performs the surveillance of CCTL either through CB and accreditation body. tasks in the following ways (SCCS, 2018a).

Communication management: Different entities as mentioned below have been involved in managing communication which are; NSC, CCTL, and CB (SCCS, 2018c).

Certificate withdrawal, revocation, and termination: The fee will not be refunded in case of termination, withdrawal or suspension of a certification procedure. Upon certificate revocation, CCTL will immediately cease the usage of CC certificate and will add details of the TOE to HPL. The CB can terminate the on-going certification process without certificate issuance by giving notice if the specified conditions are met (SCCS, 2018a). Sponsor has an authority to replace or withdraw the CCTL’s appointment by serving a written notice. The CB is entitled to accept/reject the application. The project details will then be moved to HPL and will be marked as ‘withdrawn’.

Records management: The scheme does the records management by sharing records via TKM and EPM meetings, publish ST after sanitizing the sensitive information, and sanitize reports if developer and sponsor are different entities.

Assurance continuity: AC can be of different nature including re-evaluation and certificate maintenance. When the impacts of changes made to the TOE on assurance baseline are minor, then certificate maintenance will be performed (SCCS, 2018a). Re-evaluation will be applicable if such effects are major. The sponsor will exercise impact analysis procedure and draft IAR to determine whether these impacts are major or minor. Based on it, CB will decide AC. Based on the SCCS, Table 4 presents important responsibilities of CB.

Dispute handling: Disputes are resolved through SCCS’s Complaints, Disputes and Appeals process (SCCS, 2018c; SCCS, 2018a).

Use of logos and protective marks: The sponsor can use SCCS and CC marks provided that the evaluation has been successful. These marks should be used in standard form except the alterations in monochromatic color schemes and size (SCCS, 2018a).

Netherlands scheme

Introduction: The objective of NSCIB is to certify the IT products in accordance with CC, which is composed of ISO/IEC 15408 and ISO/IEC 18045 (NSCIB, 2017).

Accreditation Procedure of the Testing Facility: The procedure for licensing the ITSEF with the CB consists of the following phases (NSCIB, 2017):

• Start of Licensing Process: Firstly, the ITSEF needs to submit an application to the CB. In case of rejection of the application by the CB, the notification should be sent to the ITSEF which should counter all the reasons to get the application accepted. In the case of acceptance, the notification informing the licensing-ID and the assigned certifier must be sent to the ITSEF.

• Kick-off Meeting: The kick-off meeting after the acceptance of the application will be arranged in which ITSEF’s knowledge regarding the licensing process is ensured by the CB, the status of ITSEF with respect to ISO/IEC-17025, evaluation of testing, and verification of the preliminary licensing work plan and draft of licensing agreement is formulated.

• Licensing Agreement: The licensing agreement is sent by the CB to the ITSEF and in case, the CB wishes to reject the licensing agreement, it can only reject within the period of 30 days after the letter is sent to the ITSEF. Only those ITSEFs are authorized to perform evaluations which are licensed by the CB (NSCIB, 2017). Table 2 presents the procedures adopted by NSCIB to accredit evaluation facility.

• Execution: The licensing plan can be subsequently executed by the ITSEF and the CB.

The flow of evaluation and certification: In the evaluation facility ITSEF, when the TOE or PP is evaluated for the first time, the process goes through three phases (NSCIB, 2017; Weith, 2014). First one is the preparation phase, which involves the submission and processing of the formal application. The major responsibilities of laboratory facility by considering NSCIB are discussed in Table 3. Second one is the monitoring phase, which involves the evaluation of product under the control of CB and results in the production of ETR. Table 4 presents important responsibilities of CB in accordance with NSCIB. Finally the certification phase incorporates the final actions which are needed to be taken and the generation of the certificate.

Surveillance: The scheme is responsible for performing surveillance activities. The NSCIB does not conduct an audit periodically after the issuance of certificate except for the reception of complaint (NSCIB, 2017). The identification of roles and responsibilities performed by the NSCIB is presented in Table 3.

Assurance continuity: In case of changed TOE, the sponsor is required to apply for re-certification or maintenance of assurance depending upon the nature of changes caused.

Dispute handling: If some dispute regarding the certification agreement cannot be resolved between the concerned parties then that can be settled according to the laws of Netherlands courts in the Hague.

Interpretations: If the assessment guidelines during evaluation demand clarity then the ITSEF, CB, and the sponsor shall find criterion interpretation which is acceptable but if no acceptable interpretation is found, the ITSEF can submit RI at CB. Moreover, if CB is unable to answer the RI, then it can make a request to other international groups of experts, ITSEFs, or CBs (NSCIB, 2017).

UK scheme

Introduction: For COTS products’ evaluation, the UK IT Security Evaluation and Certification Scheme is used which is operated by the CESG. Its CB is CESG CB, which is developed by the scheme (UK-Scheme, 2013c). The CESG CB is responsible for appointing the CLEFs to carry out products’ evaluations (UK-Scheme, 2013c; UK-Scheme, 2014b). The UK IT Security Evaluation and Certification Scheme is comprised of the SINs, IICC, UKI, UK Scheme Publications, assurance maintenance documentation, etc.

Accreditation Procedure of Testing Facility: For the two categories, the CLEF will be evaluated to give UKAS accreditation (UK-Scheme, 2013b; UK-Scheme, 2014a). First one is the permanent lab and second one is the on-site testing lab. The CLEF is needed to perform the evaluations and their respective reports in accordance with the UKAS accreditation. Table 2 presents the procedures adopted by UK scheme to accredit evaluation facility. The scope of UKAS accreditation includes use of evaluation criteria, evaluation methodology, and use of CESG Generic Test Method.

The Flow of Evaluation and Certification: The certification procedure is decomposed into the following stages (UK-Scheme, 2014a).

• Preparation: In order to determine the suitability for the evaluation of TOE, relevant information is gathered. The major responsibilities of laboratory facility given in UK scheme are discussed in Table 3.

• Evaluation and Certification Phase: Based on UK scheme, Table 4 presents important responsibilities of CB (UK-Scheme, 2016; UK-Scheme, 2013b).

• Assurance Maintenance Phase: Assurance maintenance phase is essential to have confidence that the modified TOE is secure without the formal re-evaluation.

• CWP: For outlining the certification work which needs to be carried out by the CESG CB, the CWP is used which can later be modified too. Certification activity can be review of scope of TOE, etc. (UK-Scheme, 2014a).

Surveillance: The surveillance activities are performed by accreditation body and CB. Initially, the assessment visits are carried out by the UKAS assessors. After the date of accreditation, the first surveillance visit is usually carried out after the duration of 6 months and successive visits are conducted on yearly basis. Moreover, in order to update the accreditation of an existing CLEF, extended surveillance visits are carried out after which the agreement is signed between the UKAS and CESG CB to decide the scope of extended accreditation schedule (UK-Scheme, 2014a; UK-Scheme, 2013a; TSM, 2013). Following the UKAS reassessment, the CESG CB will carry out the surveillance by showing involvement in the procedure for certification of evaluations.

Suspension or Withdrawal of Appointment: In case of the lapse of UKAS accreditation or if the CLEF breaches the appointment’s conditions, the CESG CB can give short notice to withdraw the appointment of CLEF. If the CLEF wishes to withdraw from the scheme, it must give at least 3 months prior notice and the same notice period is expected from the CESG CB if it wishes to withdraw or renew the terms of appointment of CLEF. At CLEF appointment’s termination, the CESG CB will determine whether any kind of evaluation work will be continued to let CLEF fulfill its contractual duties to its sponsors. If the scheme is to be terminated, then CESG CB can also withdraw all CLEF appointments on 6 months notice (UK-Scheme, 2013b; UK-Scheme, 2014a).

Assurance Continuity Management: The judgement of assurance maintenance for the changed TOE is based upon the IAR produced by the sponsor/developer to demonstrate the impact after changes to the TOE have been done, whereas, for the certified IT product, the assurance maintenance is assessed on the basis of ETR, CC certificate, ST, and CR (UK-Scheme, 2013b).

Dispute handling: If the dispute occurs between the CESG CB and the CLEF, then it can be resolved by the help of head of CB or the scheme senior executive.

Interpretations: The CCUKSG meeting is required to be held after every 3 to 4 months approximately (TSM, 2013, EPM, 2013; CWP, 2013). The CCUKSG meeting is focused on the discussion and agreement on UK issues, international issues, and the UK interpretation like security characteristics, CC portal, CESG website, CCRA, CC/CEM, SOGIS-MRA, ISO, cPPs, and other supporting documents.

Maturity and technical skills

US gained signatory status back in 1998, whereas UK and Netherlands started evaluating IT products in 2000. Singapore just attained signatory status and is assessing IT products at a great pace. By considering this factor, it can be inferred that the US, UK, and Netherlands are quite mature and can conduct evaluations with great technical capabilities. Similarly, the US has developed nine evaluation facilities, which are working in different domains to evaluate a diverse range of IT products. On the other hand, the UK and Netherlands have succeeded in developing two laboratories only. So far, Singapore is working efficiently and has already developed 4 evaluation facilities concerned in evaluating network and digital signature devices.

Another important factor to consider is the number of evaluated products which demonstrates the technical capabilities of a particular scheme. Till now, the US, UK, Netherlands, and Singapore have acquired first, second, third, and fourth positions respectively.

The technical competency of the scheme can also be evaluated by considering the maximum level of the product being evaluated. Netherlands, UK, and Singapore schemes possess technical skills of evaluating the products up to EAL7, EAL5, and EAL4 respectively. Another important factor under consideration is the number of developed PPs that can be used for evaluation. So far, the US, UK, and Netherlands have developed 294, 22, and 4 PPs respectively. In the US, NSA/NIAP are the lead responsible authorities that must be consulted in case of any query, whereas in the UK CESG is the responsible body. Netherlands and Singapore have nominated the Ministry of the Interior and Kingdom Relations and Infocomm Development Authority of Singapore as lead responsible bodies.

Comprehensiveness

It is concerned with the completeness of evaluation procedures and policies. It is about how well a scheme is organized and discusses details of the followed procedures.

In a UK scheme, the tasks of the evaluation facility have not been defined in detail, whereas the US scheme elaborates the tasks of the evaluation facility properly. The significant tasks to be followed are to: implement the scheme, manage legal agreements, regulate information flow, prepare technical reports, and follow phases which assist in evaluation and certification procedure. The Singapore scheme describes all legal policies, procedures, agreements, and feasibility study with regard to the evaluation facility and the necessary guidelines in the preparation of technical reports and the management of records and meetings. In contrast, Netherlands scheme only provides guidelines regarding the management of agreements and the production of technical reports. As far as the documentation guidance or the components of scheme is concerned, the UK and US schemes provide a large variety of documents including the defined national and international standards, whereas Netherlands and Singapore schemes provide limited documents. Moreover, the UK, Singapore, and US schemes involve parties like the procurement body, consultant, and TC as additional participants, whereas Netherlands scheme limits the participants’ role to sponsor and developers only.

The UK and Netherlands schemes do not specify any steps necessary for performing the evaluation and validation of an identified product whereas the US and Singapore schemes do specify. According to the US scheme, the check-in, evaluation, and approval phases refer to the steps for evaluation which depict the submission of the TOE, testing of the TOE, and final declaration regarding TOE’s status, may it be validation or non-validation. The description of steps for evaluation has been done by the Singapore scheme in detail, as it demands the need for submission of CAF to the CB. In case of acceptance, the CB sends an approval letter to the sponsor and the evaluation facility too. Immediately after that, a meeting is called by the CB which demands the need for participation from the evaluation facility to carry out reviews and generate CR.

Finally, coming over to the accreditation procedure, Netherlands scheme does not explicitly define the accreditation procedure but the UK scheme incorporates the usage of evaluation criteria, a methodology for evaluation, and CGOR for performing accreditation procedure. Moreover, the US scheme defines the accreditation procedure to be comprised of the guidelines provided by defined national standards and proficiency testing involving the initial on-site visit and review of the evaluation facility’s management system. On the other hand, the Singapore scheme demands the need from the evaluation facility to be capable of evaluations at EAL4 and the facility must be compliant with defined national and international standards.

Role of sponsor

The role of sponsor according to UK scheme demands the need to get all types of supporting documentation related to the TOE from him/her for the certification of the TOE and its assurance continuity, whereas Netherlands scheme applies the responsibility for the payment of all dues regarding the evaluation of TOE on the sponsor. Moreover, the sponsor also maintains the communication channel between the CB and the evaluation facility. The US scheme lays greater responsibility on the sponsor as he must inform if any modifications have been made to the TOE, provide access to the testing environment, clarify the scope of sensitive information of the respective ST, and answer all queries from the evaluation facility while the Singapore scheme explains the role of sponsor very briefly.

Role of CB

In defining the role of CB, the UK scheme has not played any effective role while Netherlands, US, and Singapore schemes have explicitly discussed the role of CB in detail. According to Netherlands scheme, the CB should assure the maintenance of the quality system of evaluation facility, assist the body which issues certificates in making evaluation assessments, make a decision regarding issues of CB with the third parties, and perform issuance of certificates and licenses to the sponsors and evaluation facility respectively. The US scheme discusses the role of CB by implying the responsibilities like, the smooth functioning of evaluation facility, approval, publication, and monitoring of evaluation facility. Moreover, according to the US scheme, CB should perform information sanitization, development and validation of PPs, verification of ETRs’ compliance with the developed PPs, promotion of the CC certificates, and assurance of the correct usage of scheme and logos of CC. In contrast, the Singapore scheme restricts the CB to manage the operations of scheme, deal with the certification application, supervise evaluation done by the evaluation facility, issue CR and certificate, and play role in accrediting the evaluation lab.

Role of accreditor

As far as the role of the accreditor is concerned, all the four schemes have defined it briefly. The UK scheme expects from the accreditor to check the compliance of the evaluation facility with the necessary accreditation requirements, whereas Netherlands scheme implies the responsibility of the accreditor to check the competencies of the evaluation facility as a tester as well as an evaluator. In contrast, as per the US scheme, the accreditor must approve and validate the compliance of the evaluation facility with the defined national standards. Similarly, the Singapore scheme validates the compliance level of the evaluation facility with the specified national standards.

Technical competency

It deals with technical capabilities or skillset that the scheme possesses to carry out security evaluations and their management.

The qualification levels and the skillset of evaluators have been explicitly described by the UK and Singapore schemes. The UK scheme defines three levels of evaluators which are; trainee, qualified, and specialist staff to evaluate four different categories of products. Similarly, the Singapore scheme describes its evaluators to be two staff members having practical experience, which is presently focused on the category of network devices. In contrast, Netherlands and US schemes do not define the qualification level of evaluators rather these schemes only define the number of categories their evaluators can evaluate which are, six and eleven in the case of Netherlands and US schemes respectively. The comparison is presented in Table 6.

With respect to the types of products that are being evaluated, all four schemes have different expertise based on which they evaluate different categories of products. Those list of categories with respect to the UK, Netherlands, US, and Singapore schemes can be seen in Table 5.

As far as the acceptance of PPs by the schemes are concerned, the UK and Netherlands schemes do not specify any restriction but the US scheme approves the national PPs only. Similarly, the Singapore scheme also approves the national PP, endorsed cPP, and any other CSA recognized PP. Moreover, if the TOE is not based on the aforementioned PPs, then the Singapore scheme allows evaluation up to EAL2+ but other than that, the approval by Singapore Government Agency is mandatory.

Assurance continuity

With regard to assurance continuity, the UK, Netherlands, and US schemes bound the sponsor to apply for it, in case the modifications have been made to the TOE after certification. The Singapore scheme augments the assurance continuity procedure by suggesting the use of previous evaluation work carried out to be reused in re-evaluation as it will provide assistance in case of the modified TOE.

Legal obligations

This covers legal frameworks that the entities should comply with. The legal agreements and the requisite compliance with standards vary with respect to the chosen four schemes. The UK scheme enlists the legal obligations in the form of agreements related to full or provisional appointment, national or international issues, ownership rights, UKAS assessment during the evaluation, the release of ETR in future, participants involved in the evaluation, and compliance with the UKAS, CGOR, and ISO/IEC-17025. Similarly, Netherlands scheme also describes legal agreements related to licensing, certification, and compliance with ISO/IEC-17025. Furthermore, the US scheme describes security policy, agreement for non-disclosure of proprietary information, a contract regarding documentation sharing and conformance with defined standards, and the agreement regarding the sharing of deliverables. Just like the US scheme, the Singapore scheme restricts the evaluation facility to ensure conformance to the nationally approved PPs, SAC, and SCCS, and to sign the contract for evaluation and certification, in the form of legal obligation.

Communication and engagement

To conduct evaluation successfully, communication and coordination between major entities are extremely important.

The communication and coordination of activities with respect to the UK scheme involve meeting with the POC, CCUKSG which focus on UK /international issues and national interpretations, CPR, and meeting minutes related to the EPM. In contrast, Netherlands scheme restricts the coordination activities to a kick-off meeting. Similarly, the US scheme describes the meeting of the validator with the evaluation facility so that he can understand mechanisms ensured by the facility for evaluation and record keeping. In the same way, the Singapore scheme describes task close-down meetings in which the evaluation facility provides a synopsis of evaluation tasks and the arrangement of TKM.

Surveillance

It is important to oversee evaluation tasks, legal agreements, communication, technical reports, compliance and information flow for ensuring quality and credibility of CC assessment.

The UK scheme defines surveillance in terms of monitoring of the CB and the evaluation facility by the certification scheme. Similarly, US and Singapore schemes are responsible for ensuring a smooth process in evaluation and accreditation. The surveillance by the accreditation body in Netherlands scheme has not been taken into account. In contrast, the UK scheme defines the first surveillance by the accreditation body to be carried out after a period of 6 months, whereas the successive surveillance visits are to be conducted on a yearly basis and the full assessment is to be carried out after 3.5 years with respect to the accreditation date and thereafter 4 years interval. Similarly, the US and Singapore schemes describe the procedure of audit to be conducted by NIAP and NVLAP and by SCCS and SAC respectively.

Transparency

The devised policies should be transparent and stringent in nature for ensuring accurate implementation of CC.

For ensuring transparency, the UK scheme defines information sanitization policy, withdrawal policy that implies condition on the evaluation facility to notify CB at least 3 months before carrying out withdrawal of appointment, conflict of interest policy so that the sponsor can not hire staff from CLEF within 2 years after his termination from the evaluation facility, but the scheme does not specify termination policy. Similarly, Netherlands and the US schemes define the sanitization of output by the implementation of secure ICT infrastructure, conflict of interest policy which in case of Netherlands scheme, deals with the clause that the evaluator can not develop, evaluate, or provide consultancy to the TOE, and in case of US scheme deals with the clause that the facility can not provide consultancy to the product. Unlike the UK scheme, Netherlands scheme provides termination policy too by mentioning that the evaluation facility can terminate the agreement with the CB by notifying the date prior to termination but in contrast, the US scheme does not specify the termination policy.

The Singapore scheme also defines transparency in terms of information sanitization policy and conflict of interest policy. Moreover, the sponsor can withdraw an appointment which needs to be approved by CB. The termination policy in Singapore scheme has also been discussed in detail which states that a certificate can be revoked if the developer or sponsor or evaluation facility breaches the terms and conditions, fails to disclose vulnerabilities and corrective actions, the evaluation activity has not been conducted for more than 60 consecutive days, the sponsor or the facility does not take actions in time, the evaluation scope has been changed without informing the scheme and CB, or the quality of the work performed by the facility is not up to the mark.

Validity period

As far as the validity period of the certificate is concerned, the UK scheme does not specify any period of time. In contrast, the US and the Singapore schemes define the period of validity to be 5 years. The US scheme also defines the validity period to be 5 years and additionally specifies the validity of the site’s certificate to be 2 years.

Cost

With regard to the charges levied by the CB, the UK scheme states that the charges for evaluation, reevaluation, assurance maintenance, training course, and charges for UKAS assistance must be charged. In contrast, according to Netherlands scheme, certificate investigation cost, advanced invoice, breach of conditions’ penalty, certificate cost, certification mark usage cost, and the setting up along with the maintenance of scheme cost must also be charged. The US scheme demands the cost of certification and accreditation of the facility’s cost. Similarly, the Singapore scheme charges the cost of accrediting the facility and for doing certification of TOE with respect to the assurance level.

Withdrawal procedure

The withdrawal by the CB is also defined explicitly by all four schemes. According to the UK scheme, the CB can withdraw from the facility by giving 3 months prior notice. In case the scheme gets terminated, then CB has the right to terminate the appointments of all facilities on 6 months’ notice. Netherlands scheme states that CB can reject the application for the licensing process and will accept the renewed application if all the rejection reasons are fairly justified by the sponsor. Like the UK scheme, the CB can withdraw the appointment of facility based on defined policies. Similarly, the Singapore scheme can withdraw the appointment of the facility too by giving 1 month written notice to CSA.

Certification

The usage of international certification marks in the UK scheme has not been specified. In contrast, Netherlands scheme lays down the condition to sign a joint claim with the CB and the sponsor if the latter wishes to use certification marks. Moreover, the CB must also ensure only authorized use of certification marks. The US and the Singapore scheme allow the use of their respective marks based on the defined policies.

Termination procedure

The termination agreement in the case of the UK scheme can be signed between the CB and the facility if the latter does not comply with the defined rules. Netherlands scheme can terminate the certification process prematurely if the sponsor does not, meet the plan regularly, pay dues, or provide correct reports. Moreover, if the evaluation facility breaks the rules, then its termination by the CB will be done on the final day of the month provided that 3-month notice is given to the facility. The Singapore scheme gives the CB right to revoke a certificate if sponsor, developer, or facility breaches the specified terms and conditions. Moreover, the CB can terminate the on-going certification process without the issuance of the certificate by giving notice.

Standardization

The scheme should have some standardized bodies to validate legal compliance and ensure uniformity of assessments conducted by different evaluation facilities. According to UK, Netherlands, US, and Singapore scheme, the standards as mentioned in Tables 3–5 should be followed and compliance should be made with them.

Capacity building

It is another key factor that deals with arrangement of trainings to develop skillset for conducting evaluations according to CC. The US scheme has arranged a number of workshops to develop the technical and administrative skillset to conduct evaluations complying with CC. Similarly, Singapore hosted an international conference in October 2019.

Strategic guidance

The guidance documents play a pivotal role in implementing the scheme properly. All four schemes provide some documents and forms, which can assist in developing and implementing a scheme properly. The UK and Netherlands provide templates of technical reports, security manuals and meeting minutes. Along with these templates, the US also provides policy documents and NIST handbook to further ease the procedure. Similarly, UK presents a format for meeting minutes, technical reports, and security policy complying with ISO 27001 and ISO 27002.

Structure

The structure and hierarchy of roles help in managing evaluation tasks. The US presents the structure of the scheme in a detailed form. Based on it, the scheme should has five levels of individuals in hierarchy, whereas the UK, Netherlands, and Singapore do not suggest such a structure.

With respect to the evaluation facility

Hierarchy in Evaluation Facility’s Role: It is suggested to decompose the evaluation facility’s role into multiple components so that role-based access control and separation of duty can be ensured.

Owner of the Evaluation Facility: As specified in Table 3, the evaluation facility can be government owned or some private company owned but in either case, it must comply with the national laws.

Requisite Compliance with International Standards: Those international standards should also be searched for which provide guidelines related to the chosen categories of products as are being considered in US, Singapore, Netherlands, and UK schemes. Afterward, compliance with those standards should be made as a prerequisite for certification in order to ensure better security.

Assurance of Information Sanitization: Principle of information sanitization should be followed as highlighted in all four schemes. It demands the need to sanitize the sensitive information from the product, before making it public or handing over it to the developer respectively as mentioned in UK Scheme. Moreover, the information security policy for implementing the non-disclosure of proprietary information is also recommended as is being followed in US scheme. The secure infrastructure of evaluation facility should also be ensured for information sanitization which is referred Table 3.

Availability of Security Manual: For providing assurance that appropriate security has been implemented inside the evaluation facility, presence of security manual is mandatory as specified in Singapore Scheme.

Qualification Levels of Evaluator: The evaluation facility’s policy must define the qualification levels of an evaluator as defined in UK scheme so that the product’s evaluation can be assigned to the respective evaluator based on his qualification accordingly.

Legal Agreements: Legal agreements like contract between sponsor and CCTL, certification agreement with CB, contract regarding sharing of technical reports as mentioned in subsection of Singapore and UK schemes, subsection of Netherlands scheme, and subsection of US scheme respectively, should be signed in order to avoid any conflict in the future. In case, the facility terminates the agreement with the CB, it should make the required procedures and formalities clear.

Steps for Evaluation: The steps followed for evaluation like pre-evaluation, check-in meeting, monitoring, and certification phase should be distinctly mentioned as specified in US scheme so that CB can ensure that appropriate steps are being followed.

Tasks of Evaluation Facility: It would be significant to mention the tasks of an evaluation facility like preparation of technical reports, regulation of information flow etc. as elaborated in Table 3 so that no essential task is missed by it.

Withdrawal by Evaluation Facility: There should be some defined period of notice, which is 3 months in case of UK scheme as defined in Table 3, given by the evaluation facility before withdrawing some agreement with the CB or the developer so that they can take appropriate actions accordingly.

Records and Procedures Meetings: Records and procedures meetings should be held by the evaluation facility with the CB as mentioned in Netherlands scheme, with the sponsor as described in Singapore Scheme, and with the validator as mentioned in US Scheme, in order to ensure the correct implementation of all procedures and to inform sponsor about the evaluation progress of the product respectively.

Conflict of Interest: The facility should ensure that none of its members get a chance to assist the sponsor or developer as highlighted in UK scheme in the evaluation of product and it should assign a time period after the termination of its employee to not serve any developer or sponsor.

Assurance Continuity Management: In case the modifications are made to the certified TOE, two verdicts can be given by the evaluation facility, which are major and minor revisions as specified in US scheme. In case of minor revision, the facility will reevaluate it by itself while in case of major revision, it will not only reevaluate it but it will also forward the ETR to the CB.

With respect to CB/validator

The important practices that should be followed by CB to perform successful evaluation are listed below.

Role of CB: It should be responsible for managing the operations of scheme by enforcement of defined policies in accordance with CCRA as described in UK scheme, issues certificates in making evaluation assessments, and provide licenses to evaluation facilities as defined in scheme of Netherlands. Moreover, it also develops PPs, promotes integrity of CC certificates, etc. which has also been clearly defined in Table 4.

Charges Levied by CB: In order to ensure the correct implementation of regulations, the CB should levy charges on the evaluation facility. Moreover, it should also charge for the procedures like, certification, re-evaluation, assurance maintenance, accreditation, etc. as defined in Table 4.

Interaction of CB with Evaluation Facility: Healthy interaction should exist between CB and evaluation facility so that they both remain at the same level in following the procedures. The hiring of staff for evaluation facility by the CB, management of documents to inform the CB regarding the evaluation facility’s progress, and the CB’s task to resolve the queries related to CC interpretation, etc., as mentioned in UK and US respectively, are the tasks which are included in the interaction between CB and evaluation facility.

Assurance Continuity Management: In case the modifications are made to the certified TOE, two verdicts can be given by the evaluation facility, which are major and minor revisions as described in US scheme. In case of minor revision, the CB will not perform any further action. In case of major revision, the CB will prepare the VR and will add the revised version of the product to the PCL.

Notification of Changed Status: As elaborated in Singapore scheme, the CB should notify the community of any changes to the status of approved CC laboratories.

Surveillance by CB: The CB should have the authority to conduct an assessment of the evaluation facility at any time, may it be announced or unannounced, to know about its operating conditions and the methodology it follows. This has been explained in Table 4.

Publication Responsibility: The CB should publish publicly-releasable certification reports as described in Singapore scheme and periodically update a validated products list.

Guidance from Industry: The CB can seek guidance from industrial experts, if required.

Withdrawal by CB: If the evaluation facility does not comply with the defined policies and procedures then the CB should have the right to withdraw its agreement by giving it a prior notice of some specified period of time. The specified time in case of UK scheme as mentioned in Table 4 is 3 months.

Assurance of Consistency: The CB should ensure consistency of CCTL evaluations across the scheme.

Dispute Handling: In case some dispute occurs between the CB and the scheme or the evaluation facility, there should be some third party (the third party in case of Netherlands scheme is national court as specified in Table 4) which should resolve the dispute by keeping in view the interest of all stakeholders. This is distinctively mentioned in US scheme too.

Validation Procedure: For the validation of the evaluation facility, the required formal interaction between the involved or concerned entities, procedures, documentation, etc. should be previously defined. This requirement is referred from Singapore and US scheme of Table 4.

Record Sharing Means: For the secure transportation of reports and other documentation, the CB should define the policy for sharing record among multiple entities. The email policy should be defined in the case of UK scheme and certified email should be availed in case of Netherlands scheme.

Termination Agreement: The CB should be able to terminate the agreement based on the occurrence of unavoidable factors and should highlight those factors in a legal agreement with the evaluation facility as highlighted in Table 4.

Usage of International Certification Marks: As described in subsection of Netherlands scheme, the CB should sign a joint agreement with the sponsor of the certified product and the certified evaluation facility to specify the necessary conditions under which they are authorized to use international certification marks.

Steps for Evaluation: The CB should design its own method, as specified in subsection of US scheme, to ease the process of product’s evaluation which can then be used by the evaluation facility.

Requisite Compliance with Standard: If some national standard exists then CB should comply with it, provided that the requirement is given by the scheme.

With respect to certification scheme

Following are the some important role and responsibilities of scheme to conduct security evaluations with respect to CC.

Management of Scheme: There should be some government-owned body which supervises the operation of the scheme. This requirement has been inferred from subsection of US and UK schemes.

Structure of Scheme: There should be a specified hierarchy in the scheme in order to ensure the role-based access control and separation of duty. Moreover, introduction of hierarchy will help all people know their extent of role and authority.

Components of Scheme: The scheme should comply with regulatory bodies, international or national standards, legal requirements, etc. to match the national and international legal requirements. This requirement has been deduced from Table 5.

Participants of Scheme: The scheme has the right to decide the entities which can become the participants of scheme. The sponsor, developer, vendor, procurement body, accreditor etc. can be the participants as specified in subsection of UK scheme.

Role of Sponsor: The role of the sponsor should be decided by the scheme. Normally, the sponsor is responsible for supplying all documentation and the supporting material which may be required in the evaluation of the product, pays all dues on the behalf of the developer as defined in subsection of Netherlands scheme, and inform the evaluation facility if the certified product is modified. Moreover, there can be some additional role of sponsor too but it depends on the decision of scheme.

Meetings Related to National or International Issues: The scheme should specify some time interval, which is 3 to 4 months in case of UK scheme, after which meeting should be held among all the involved entities in relation to a scheme to resolve the controversial issues, may these be regarding some interpretation, some need of adding a new rule to the scheme, etc.

Management of New Interpretations: The scheme should define the proper channel which needs to be followed for adding new interpretation, if accepted, to the scheme. That proper channel can consist of application for request of interpretation followed by meeting with group of experts to decide whether new interpretation should be introduced or not as specified in the subsection of Netherlands scheme.

Certificate Validation Period: The certificate validity’s duration should be mentioned by the scheme to prevent unauthorized usage of certificate. Moreover, this duration will guide the evaluation facility and sponsor to get the laboratory and product recertified respectively. The certificate validation period for a product is 5 years and for a site is 2 years as mentioned in Netherlands scheme.

Types of Evaluated Products: Instead of making efforts to make the evaluation facility certified for all category of products, rather it should go for only those types of products whose percentage of development within the country or parent company is significantly greater in number. This practice is being implemented in different schemes with respect to countries which can be seen in Table 5. This will not only save time but it will also help in increasing the revenue that will be obtained from evaluation. Afterward, the evaluation facility should struggle to be proficient in the specified categories of IT products.

Nationally or Internationally Approved PPs: The scheme should define whether it will certify the products whose ST will be made on the basis of national PPs, international PPs, or both. Acceptance of national PPs is recommended in US and Singapore schemes.

Surveillance by Scheme: The scheme should be authorized to conduct surveillance of the evaluation facility, may it be announced or unannounced, depending upon the decision of scheme.

Assurance Continuity Management: In case the modifications are made to the certified TOE, two verdicts can be given by the evaluation facility, which are major and minor revisions as described in US scheme. The verdict of major or minor revision will be made on the basis of the guidelines and procedures defined by the scheme.

With respect to accreditation body

The roles and responsibilities that the accreditation body should perform are enlisted below.

Role of Accreditation Team: The roles of accreditation team should be distinctively defined. It can be composed of technical manager, coordination manager, quality manager, etc.

Role of Accreditor: The role of the accreditor is to approve and verify the compliance of the evaluation facility with the scheme and the specified national and international standards.

Accreditation Procedure: The accreditor should consult all specified standards and procedures prescribed in the scheme to accredit labs. The accreditation procedure should involve proficiency testing, review of the management system of laboratory, on-site visits, etc. as mentioned in US scheme, to get complete assurance that the laboratory is operating as specified.

Surveillance by Accreditation Body: The accreditation body should either specify the time interval after which it will conduct surveillance to accredit the evaluation facility or should visit the facility without making the visit announced in order to validate the procedures being followed in the facility for evaluation of products. It is recommended that the first surveillance should be conducted after period of 6 months as done in UK scheme. It is also suggested to conduct the successive surveillance after a period of 1 year and to conduct full assessment after a period of 3.5 years.