PR110112 - Annex 5 - Tender Evaluation Matrix Notes v1.0 - STFC's ...

PROTECT -­‐ CONTRACT ICE-­‐CSE Computational Science & Engineering Department Science & Technology Facilities Council Annex 5 -­‐ Tender Evaluation Matrix -­‐ Notes 1 Introduction These notes are to provide additional background and clarification of the criteria specified in the tender evaluation matrix. The tender evaluation will be performed on the responses to the Mandatory and Desirable requirements defined in the evaluation matrix, marked Mx and Dx.x respectively in the matrix. All potential suppliers who comply with all the Mandatory requirements in the evaluation matrix, and who also comply with the other mandatory criteria in the “Selection and Award Criteria” documentation, will have their tender responses evaluated against the scoring matrix. We have deliberately aimed to state our requirements and it is the purpose of the open tender exercise that the supplier who best meets our requirements (as determined by the scoring mechanism) will be the most appropriate partner in the research collaboration. Mandatory means Mandatory: any response which does not comply with all the Mandatory requirements will not be scored. Points will be awarded for meeting some or all of the level of the Desirable requirements detailed. For each requirement, a point score in the range 0-­‐5 will be awarded, and then multiplied by the weighting factor defined to reach a total point score for each response received. All compliant tender responses will be scored on the basis of the written submission. All suppliers who provide compliant responses will be invited to present their solutions to us and respond verbally (and subsequently in writing) to clarification questions, and the point score for each supplier will be revised as we see appropriate based on the presentation and question/answer session. The supplier with the highest point score following this re-­‐grading exercise will be awarded “preferred supplier” status, and all other suppliers will be informed that they are not “preferred supplier”. The specification is divided into 5 major sections: Section Brief Description Max Score Normalised (100%) Environmental 20 8.2 ICE-­‐CSE BASE Provision of an HPC system and file store 35 14.3 ICE-­‐ CSE Tender Evaluation Matrix – Additional Notes Page 1

PROTECT -­‐ CONTRACT ICE-­‐CSE ADVANCE Provision of an “experimental” system on the suppliers roadmap to Exascale and forms the basis of the collaboration 75 30.6 ICE-­‐CSE FUTURES Access to future products and input by STFC into the suppliers Exascale computing developments 25 10.2 ICE-­‐CSE APPS Establishing a collaborative partnership with the supplier 90 36.7 Total 245 100.0 2 Environmental (M0, D0.1, M1, M2, D1.1, D1.2, D1.3) This section refers to the complete environmental specification of the solution, hence applies in total to the ICE-­‐CSE BASE system, the ICE-­‐CSE ADVANCE system and the file storage system which works with both systems. The £6m total figure is inclusive of all taxes, most especially VAT, and covers the entirety of the tender response. We are not asking for a breakdown of spending by system or by component. Our power budget stated is derived from a limit we have on recurrent operating costs. We have translated our budget figure into an average 300kW, but are asking suppliers to quote within the peak envelope of 400kW, and are taking the risk on electricity price fluctuations ourselves. STFC are commissioning water cooling systems in the Daresbury machine room which the supplier will be required to use to cool both the ICE-­‐CSE BASE and the ICE-­‐CSE ADVANCE systems. The figure of 75% (or greater) of the heat expelled by the equipment to be cooled by water accounts for heat expelled to air and chilled by CRAC units by both compute and storage equipment. We are very interested in high-­‐temperature cooling options, which allow us to reduce running costs by minimising the requirement to operate water chillers. We are also interested in reducing infrastructure costs by using a single chilled water loop which passes through the computer equipment and also expels air externally, and therefore needs to be include anti-­‐freeze. Suppliers should satisfy themselves that they have included all the necessary pipes, heat exchanging equipment etc (see M2) to connect and utilize the STFC water supply. 2.1 Additional requirements Suppliers should satisfy themselves that the equipment can be delivered to and installed in the computer hall using the designated access route. (See M2.) Suppliers must provide a risk assessment and method statement for delivery and installation of equipment in advance for approval by STFC. Responses for the ICS-­‐CSE BASE and ICS-­‐CSE ADVANCE systems may differ. 2.2 Hardware Support The Supplier shall specify the proposed hardware support arrangements for the initial 3 years including: • Nominated providers of support for all aspects of the hardware • Contact details for the nominated providers • Hours of support throughout a calendar year • Responsibility and capability for fault diagnosis ICE-­‐ CSE Tender Evaluation Matrix – Additional Notes Page 2

PROTECT -­‐ CONTRACT • Response times for hardware diagnosis and replacement • Responsibility for delivery, fitting and despatch of parts • Policy on EOL hardware disposal • Escalation procedure 2.3 System Software Support The Supplier shall specify the proposed system software support arrangements for the initial 3 years including: • A definition of the scope of system software • Nominated providers of support for all aspects of the system software • Contact details for the above • Hours of support throughout a calendar year • Responsibility and capability for fault diagnosis • Mechanism for application of fixes and upgrades • Responsibility for installation of software upgrades • Escalation procedure • Tools and interfaces to allow monitoring of the hardware status, including peripherals, and system software 3 ICE-­‐CSE BASE (M3, M4, D4.1, D4.2, M5, M6, D6.1, M7, D7.1, M8, D8.1, D8.2) ICE-­‐CSE BASE is a “conventional” HPC system, on which we want to be able to engage industrial partners and run our current suit of HPC applications. 3.1 Permanent File Store Whilst the file store requirements are specified in the ICE-­‐CSE BASE section it is emphasised that the file store must also support the ICE-­‐CSE ADVANCE system, and should be sized and configured to support both ICE-­‐CSE BASE and ICE-­‐CSE ADVANCE systems and will be expected to meet similar performance requirements with each system as detailed in the IO benchmark requirements. 3.1.1 Performance The assessment of file store performance is based upon the ratio (rp) rp= aggregatebandwidthtotalmemoryWhere • aggregate bandwidth is in bytes/second • total memory is in bytes A minimum rp of 1/250 is mandatory. The aggregate performance will be measured using IOR, running on all nodes in both systems at the same time. The benchmark file will be provided separately, but will run a set of tests at block sizes from 1MB to 512MB, with transfer sizes from 64k to 1024k. 3.1.1.1 Minimal systems example Suppose ICE-­‐CSE BASE has 2048 cores each with 1GB memory and 2 X 256GB large memory nodes. Total memory = 2,560GB ICE-­‐ CSE Tender Evaluation Matrix – Additional Notes Page 3

PROTECT -­‐ CONTRACT Suppose ICE-­‐CSE ADVANCE has 16,384 cores each with 1GB. Total Memory = 16,384 GB Total memory = 8,192 + 2,560= 18,944 GB Minimum combined aggregate performance should be 10,752/250 = 76GB/S Minimum usable storage capacity = 10 x Total Memory = 10x18,944GB = 189,440 GB which is less than the 200TiB mandatory, therefore usable storage = 200TiB. 3.1.1.2 Maximal Systems (or Mandatory Plus all desirables) ICE-­‐CSE BASE: 3072*1GB + 4x 512GB = 5,120GB Memory ICE-­‐CSE ADVANCE: 32,768 cores x 1GB/core = 32,768GB Memory Total Memory: 37,888GB Minimum combined aggregate performance should be 37,888GB /200 = 189 GB/S Minimum usable storage capacity = 15x Combined System Memory = 15x37,888 = 568,320GB 3.2 Power Consumption We are keen to minimise operating costs and believe this can be accomplished if we are able to monitor and control power usage down to the level of single parallel jobs. Tools which enable us to perform this task are therefore desired. 4 ICE-­‐CSE ADVANCE System (M9, D9.1, D9.2, D9.3, D9.4, M10, M10.1, M11, D11.1, D11.2, D11.3, M12) The ICE-­‐CSE ADVANCE system should represent “state of the art” technology in the supplier’s roadmap along the path to Exascale computing. We wish to use this system to • demonstrate extreme parallel scaling and performance over a large number of nodes, • to optimise existing codes in order to enable them to run on future Exascale systems and • to develop new codes where existing codes have reached the limit of their capabilities. 5 ICE-­‐CSE FUTURE (M13, D13.1, D13.2, D13.3) The reason for establishing the ICE-­‐CSE collaboration is to give us access to new technology developments when they become available. We recognise that we cannot pre-­‐pay for future hardware, and therefore do not require that new hardware be delivered to us on site in Daresbury, but we want to define part of the collaboration as giving us access to this sort of equipment. We also want to be able to contribute to our partner’s product development plans, so having access to pre-­production equipment and a feedback route back into product development will both help enable this. 6 ICE-­‐CSE APPS (M14, D14.1, D14.2, D14,3, D14.4, D14.5, D14.6, D14.7, D14.8) We ask that the supplier prepare a business plan, governance structure and examples of similar collaborations in which various aspects of the collaboration might be included. We intend that this collection of requirements is consistent with information CSED has published: that the “collaboration” aspect of ICE-­‐CSE is vitally important to us and that we intend to recognise input in this area significantly in determining our preferred supplier. ICE-­‐ CSE Tender Evaluation Matrix – Additional Notes Page 4

PROTECT -­‐ CONTRACT The ICE-­‐CSE collaboration should be viewed as a stepping stone to a longer term and larger venture, which we have been discussing for a number of years. We continue to seek additional funding and urge collaborators to think of how the role of the ICE-­‐CSE collaboration could be expanded with additional funding. 6.1 D14.8 Additional capital and recurrent funds are most likely to come from the Department for Business, Innovation & Skills (BIS). Typically this must be matched by investments from other partners. Suppliers might like to consider initiatives such as Technology Innovation Centres (TICs) and their funding models. ICE-­‐ CSE Tender Evaluation Matrix – Additional Notes Page 5