4th International Industrial Supercomputing Workshop Supercomputing for industry and SMEs in the Netherlands

Transcription

1 4th International Industrial Supercomputing Workshop Supercomputing for industry and SMEs in the Netherlands Dr. Peter Michielse Deputy Director 1

2 Agenda q Historical example: oil reservoir simulation q Funding model of the national e-infrastructure Ø The question of why q SURFsara services q Approach to industry and SME Ø Under what conditions q Challenges Ø The question of who Ø The question of how q Actual examples Visit CSC

3 Oil reservoir simulation q Major activity for large oil companies q In the 1980 s, HPC was equivalent to vector computing q However, adaptive numerical techniques could not efficiently being implemented on vector machines 3

4 Oil reservoir simulation q So, what happened: Ø Next to their vector systems, oil companies bought parallel systems Ø Expertise was developed internally Ø Funding of and collaboration with universities to solve the problem q The message here is: Ø Multinational companies fill their own needs Ø Collaboration with universities gives access to systems and expertise q But what about smaller companies? 4

5 The way forward q Many reports have appeared on the need for broad HPC access for industry at large: Ø Council on Competitiveness in the US Ø IDC reports Ø EC communications on HPC and the market Ø PRACE and its pillars Ø National reports Ø q In the Netherlands, this has resulted in co-funding of the national e-infrastructure by the Ministry of Economic Affairs 5

6 Infrastructure Innovation with Impact q 34 M value program ( ) q 7 M funded by Ministry of Economic Affairs q Objective: Ø Providing competitive advantage with the national ICT-infrastructure for Research Ø Technology transfer from science to business in this area is crucial for knowledge economy in the Netherlands Investments in keeping the national e-infrastructure at the competitive level Access to and support for the integrated e-infrastructure for industry and SMEs 6

7 Dutch Top Sector Approach q The overall connection is to the so-called Top Sector strategy to improve the Dutch economy (focus on SMEs) q Innovation Ø through ICT Ø of ICT q ICT importance for all sectors q Technology transfer from science to business q Budget: 1B 7 7

8 Technology Transfer Approach SURFsara q High-end ICT services q Academic and precompetitive activities q Innovation in HPC q From state-of-the-art to a Proof-of-Concept (PoC) q Customers: research & education (universities, research institutes, businesses) 8 E.g. Vancis, Vortech q Commercial ICT services and support q Market activities q Innovation by HPC q From PoC to business integration q Customers: various industries, life sciences & healthcare, public sector

9 Mission of SURFsara SURFsara supports research, educa2on and industry in the Netherlands by development and deployment of HPC- infrastructure, services and exper2se The support is guaranteed by: deployment of an integrated ICT-research infrastructure provision of services in computing, data, visualization, networking, cloud and e- science making available multidisciplinary expertise and support in ICTtechnology and applications; performing all required innovation, engineering and development activities Advanced infrastructure, service, support, innovation and development go hand in hand 9

10 SURFsara HPC ecosystem 6 Categories of Services q Low-latency, high-bandwidth national supercomputer q Capacity compute cluster with capacity computing services q IAAS, PAAS and SAAS ondemand Cloud services q Loosely coupled compute and data Grid farms q Big Data cloud and software Services q Special-purpose architectures (accelerators), running hybrid computing services

11 Cartesius National Supercomputer Bull Bullx HPC cluster phase Tflop/s (now) phase Pflop/s (2014) NVIDIA GPUs early 2014 Grid Resources & mass storage 5000 Cores, 4 PB disk, 6 PB tape (BiG Grid infrastructure) 11 BioInfo Sites Life Science Grid High Energy Physics, Astronomy, BioInfo LISA National Compute Cluster Dell cluster, 6528 cores, 46 TFlop/s, Intel Xeon Phi Visualization Collaboratory High resolution Tiled Panel Display Remote Visualization, render cluster Innovative Infrastructures HPC Cloud Hadoop Beehup 11

12 Practical implementation q Under what conditions can we offer access and expertise to industry and SMEs? Ø Transparency and equality principles Ø Not available as market service Ø Pre-competitive Ø Real, market-proven costs Ø Public announcement of access and support opportunities Ø Laid down in an Agreement of Execution between Ministry and SURF (which is the umbrella organisation of SURFsara) 12

13 And then. find the SMEs q First level of challenges: Ø Where to start? High Tech Systems and Materials, Life Sciences, Creative Sector are areas with contacts Water, Chemistry as traditional users of HPC Ø How to get in touch? Find a connecting link which oversees SME landscape Syntens to organise workshops which bring together SMEs and SURFsara Previous researchers (PhD students, post-docs) have found employment in the water and chemistry sectors this is a typical example of the benefits of HPC education to economy Of course, web sites etc. Personal contacts to point to the programme 13

14 Get them look behind their horizon q Second level of challenges the missing middle: Ø Increasing hardware complexity (CPUs and GPUs) Ø Software portability and usability Ø Software scaling Ø Software licensing Ø Security and trust Ø Lack of expertise q Can be technically solved by expertise, effort, open source codes, secure and private environments needed is to convince the SMEs and the ISVs of the long-term benefits 14 The Missing Middle

15 Suppose the SME is ready q Third level of challenges - where to apply? Ø National: Through this program And still through joint proposals with universities in the regular access procedures Ø International: Through PRACE initiatives (both Tier-0, DECI, SHAPE) Through Fortissimo Through XSEDE (Industrial Challenge Program). q We may want to have some alignment here 15

16 HPC Industrial Applications: Modeling steel TATA Steel 16

17 HPC for the Chemical Sector q Chemical simulation software through SCM q SCM is a software company that develops and markets ADF and BAND simulation software q ADF (Amsterdam Density Functional software) is widely used in academia and industry q Scaling experiments on SURFsara systems q ADF and BAND can be used by Dutch scientists on SURFsara systems without fee 17

18 Water Management: Deltares q Water is key to the Netherlands and to the Dutch industry! q Many engineering companies are doing something with water: Ø Coastal models, sand motion, sea level rise, river flooding, dike simulations, wave models, harbor design, q Deltares develops many software packages for this kind of engineering work q At SURFsara, Deltares is able to: Ø Validate large model tests Ø Investigate scaling of their applications (through testing the computational kernels) 18

19 Water management: Meander q Meander, consultancy company for the Dutch Ministry of Infrastructure and the Environment q Project Maaswerken: improve safety (less flooding), ecology and transport on the river Maas q 2D simulations using WAQUA that models water levels, water streaming profiles and sediment concentrations in rivers, lakes and seas. 19

20 There is more than traditional supercomputing High Performance Platform Virtualized Everything Software, Services Support HPC Cloud Offering Fast CPU, High Memory, Large Storage and High Bandwidth Virtual Private Clusters and Networking Resource Deployment portal, Collaboration Portal User in control Powerful Flexible Versatile HPC Cloud 20

21 Hadoop Big Data Services q Open source Apache Hadoop along with a framework for map reduce jobs over data q Move processing to the data q Seamless scalability q 2009: Pilot Hadoop on Cloud q 2010: Test cluster available for scientists q 2011: Funding for production services PoC: 20 users q 2012: Production started with 72 machines * 8 cores / 8 TB storage / 64GB RAM, 3 devops, team of consultants q Lot of interest from science and business 21

22 Machine learning: Infrawatch Hollandse Brug q Large amounts of Structured & unstructured data captured at very high rates (sensors, streams, social media, ) q Creating massive historical archives q Data is more valuable in its fresh state q Archives valuable for mining patterns, trends and relationships 22

23 Conclusions q Recognition of industry and SME needs q E-infrastructure access and expertise are available q Large potential of SMEs that should benefit q Work ahead in approaching SMEs, upgrading and releasing hurdles q More alignment of national and international initiatives 23