Download This PDF! - GOES-R

ESPDS Scalable and Secure
Infrastructure
Ensuring the NOAA/NESDIS Environmental Satellite Processing and
Distribution Capabilities Meet the Growing User and Data Demands of Today
and Tomorrow
Rich Baker
Solers, Inc.
ESPDS Development Chief Architect
2013 AMS Annual Meeting

What is ESPDS?
●
●
●
●
ESPDS: Environmental Satellite Processing and Distribution System
●
Developed by the NESDIS Office of Systems Development (OSD), with Solers (“Team
Solers”) as the development contractor
● Will be operated by the NESDIS Office of Satellite and Product Operations (OSPO)
Modernizes the NESDIS Environmental Satellite Processing Center
(ESPC)
●
●
●
Single enterprise solution that meets the needs of existing (legacy), Suomi NPP, JPSS, and
GOES-R, with scalability to meet future environmental satellite needs
●
No more stovepipes!
Includes modernization of the Ingest, Product Generation (PG), Product Distribution (PD),
and Infrastructure segments of the ESPC
Provides environmental satellite data and services to a growing user community including:
●
●
NOAA Line Offices (NWS, NMFS, NOS, NIC, NESDIS, etc.)
DoD (AFWA, NAVO, etc.)
● Other U.S. and international users (government agencies, universities, foreign
partners, etc.)
Will be implemented at the primary and backup ESPC sites:
●
●
Primary ESPC site is the NOAA Satellite Operations Facility (NSOF) in Suitland, MD
Future ESPC backup site is the Consolidated Back-Up (CBU) facility in Fairmont, WV
Provides a scalable and secure infrastructure as a foundational building
block upon which all other system functions reside
© 2013 Solers, Inc.
2

Traits of a
Scalable Infrastructure
●
●
●
●
No Single Point of Failure
●
Redundancy and fault tolerance as key design tenants throughout
Line Replaceable Units
●
Can upgrade or replace existing hardware and software components
without impacting operational availability
Business Process Flexibility and Extensibility
●
Can change existing business processes within the system, and
integrate new business processes into the system, without impacting
operational availability
Horizontal Scalability
●
Can add additional hardware resources (computing, network, storage)
and software business processing instances without impacting
operational availability
© 2013 Solers, Inc.
3

Traits of a
Secure Infrastructure
●
●
Complies with applicable IT security policies,
procedures, and controls:
● NIST SP 800-53
●
●
●
●
DOC/NOAA IT Security Handbook
Center for Internet Security (CIS) Benchmarks
DISA STIG
Etc.
Provides a “defense-in-depth” foundation for
securing the system that includes:
●
●
●
●
●
Network security
Centralized identity/account management, authentication, and
authorization
Host-based intrusion detection and prevention
Anti-malware
Integrated monitoring, logging, and reporting (Security Incident and
Event Management [SIEM])
© 2013 Solers, Inc.
4

ESPDS Scalable and Secure
Infrastructure
• Suomi NPP and JPSS (via
IDPS)
• GOES-R GS PD
• Non-NOAA Satellites
(MSG, MTSAT, INSAT)
• Ancillary Data Providers
• Legacy GOES
• Legacy POES
• Future Missions
Satellite Ingest
Computing Cluster
Scalable x86 hardware cluster
with specialized adapters to
interface with satellite antenna
systems and perform RF/IF to
IP conversion of the data
• Resource Management
• Communications Framework
• Logging & Reporting
• Monitoring
Product Generation
Computing Cluster
Scalable x86 hardware cluster
that leverages a grid computing
scheduler to perform PG
algorithm execution and report
applicable status/metrics
Common Infrastructure Services
• Identity/Account Management
• HIDS
• Anti-Malware/HIPS
• Network Management
Converged 10Gb IP Networking
Virtualized
Computing Cluster
Scalable x86 hardware cluster
that hosts the distribution and
access, PG management,
common infrastructure, and
other services as Virtual
Machines (VMs)
• Database
• Data Intake & Transmission
• Scheduling
• System Backup
Enterprise Shared Storage
Scale-Out Enterprise Network Network Attached Attached Storage Storage (NAS) (NAS) solution solution with with standard standard IP-based IP-based file file access access protocols protocols (NFS, (NFS, CIFS, CIFS, HTTP, HTTP, FTP) FTP)
(EMC Isilon)
Includes switches, firewalls, and Network IDS components
(Cisco)
• NOAA Line Offices
• DoD
• CLASS
• Other U.S. and
International Users
• Ancillary Data
Users (PG Systems)
© 2013 Solers, Inc.
5

Common Infrastructure Services
●
The following slides provide an overview of the
Common Infrastructure Services depicted in the
previous diagram
●
●
●
●
●
●
●
●
●
●
●
●
Resource Management
Communications Framework
Logging & Reporting
Monitoring
Identity/Account Management
HIDS
Anti-Malware/HIPS
Network Management
Database
Data Intake & Transmission
Scheduling
System Backup
© 2013 Solers, Inc.
6

Resource Management
●
Technologies Used
●
VMware vSphere/ESXi, vCenter, and Orchestrator
© 2013 Solers, Inc.
7

Communications Framework
User
/Operator/
Admin
HTTPS
(S)FTP(S)
Load Balancer
(S)FTP(S)
Client
(S)FTP(S)
VM
(S)FTP(S)
Server
GOES-R
GS PD
WS Client
Other
System
WS Client
ESPDS
SOAP over HTTP
HTTPS
Portal
Portal
(S)FTP(S)
(S)FTP(S)
(S)FTP(S)
Server
Server
(S)FTP(S) Other Other
(S)FTP(S)
PDA
Client
Client
Service ESPDS
Service
Load Balancer
SOAP over HTTPS
VM
SOAP over HTTP(S)
ESB
ESB
SOAP over JMS 1.1
JMS Broker
VM
SOAP over
JMS 1.1
JMS Broker
VM
SOAP over JMS 1.1
Application
Server
ESPDS
Service
VM
Application
Server
ESPDS
Service
VM
Application
Server
ESPDS
Service
VM
…
●
Technologies Used
●
●
●
WSO2 ESB and Application Server
Apache ActiveMQ Java Message Service (JMS) Broker
Red Hat Linux Virtual Server (LVS) Load Balancer
© 2013 Solers, Inc.
8

Logging & Reporting
Windows Event logs
(WMI)
Microsoft Windows
Layer 3 Switch
(Cisco)
Firmware logs
(Syslog)
Monitoring
(SolarWinds Orion)
Directory Server
(Microsoft Active
Directory)
vCenter, ESXi, Resource
Coordinator, Red Hat
Repository logs
(WMI)
Resource
Management
(VMware vCenter)
Computing HW
(Cisco UCS)
Firmware logs
(Syslog)
SolarWinds Orion logs
(WMI)
Windows Event logs
(WMI)
Apache SSHD/FTPD logs
(Rsyslog)
Data Intake/Data
Transmit
([S]FTP[S]
Server/Client)
NAS Storage
(Isilon)
SAN Storage
(EMC VNX)
Firmware logs
(Syslog)
Firmware logs
(Syslog)
Linux OS logs
(Rsyslog)
Red Hat Enterprise Linux
Administrator
Portal
Logging & Reporting
(Tripwire Log Center)
Web server logs
(Rsyslog)
User
Portal
Custom Java Service logs
(Rsyslog)
Other Java
Components
(e.g. Subscription,
Product Tailoring, Ad-
Hoc search)
WSO2 ESB, WSO2 AS,
Red Hat LVS logs
(Rsyslog)
Oracle logs
(Rsyslog)
Communications
Framework
(WSO2, ActiveMQ,
Red Hat LVS)
Database
(Oracle RDBMS)
●
Technologies Used
●
Tripwire Log Center
●
Rsyslog (Linux-based syslog client)
●
Windows Management Interface (WMI)
© 2013 Solers, Inc.
9

Monitoring
Microsoft Windows
resource and service status
(WMI)
Microsoft Windows
Layer 3 Switch
(Cisco)
Computing HW
(Cisco UCS)
SAN Storage
(EMC VNX)
NAS Storage
(EMC Isilon)
Interface Status and
Bandwidth Usage
(SNMP)
Blade
Resource
Utilization
(SNMP)
I/O Data and Storage
Usage (SNMP)
I/O Data and Storage
Usage (SSH)
Red Hat OS resource and
service status
(SNMP)
Red Hat Enterprise Linux
Administrator
Portal
Logging and
Reporting
(Tripwire Log Center)
Service/Process Status and
Resource Allocation
(SSH/RMI)
Directory Server
(Microsoft Active
Directory)
Monitoring
(SolarWinds Orion)
User Statistics
(SSH)
User
Portal
Web Interface
Authentication
(LDAPS)
Service/Process Status and
Resource Allocation
(SSH/RMI)
VM CPU, Memory, and
Network performance
measurements
(SNMP)
Other Java
Components
(e.g., Subscription,
Product Tailoring, Ad-
Hoc search)
Connection Status and
Transfer Rate
(SNMP)
Service/Process Status and
Resource Allocation
(SSH/RMI)
Service/Process Status and
Resource Allocation
(SSH/RMI)
Resource
Management
(VMware vCenter)
Data Intake/Data
Transmit
([S]FTP[S]
Server/Client)
Comm Framework
(WSO2, Red Hat LVS)
Data Management
(Oracle RDBMS)
●
Technologies Used
● SolarWinds Orion Network Performance Monitor (NPM) and Application Performance Monitor (APM)
● Red Hat Simple Network Management Protocol (SNMP) Agent and Secure Shell (SSH) Server
● Windows Management Interface (WMI)
© 2013 Solers, Inc.
10

Identity/Account Management
●
●
●
●
●
Centralized identity and account management
solution
Manages human user accounts (internal and
external users, operators, administrators)
Manages machine and operating system accounts
Provides Kerberos and web services-based
authentication and authorization services
Compatible with NOAA/NESDIS HSPD-12 solution
(DoD CAC PIV token, X509 PKI certificates)
●
Technologies Used
● Microsoft Active Directory
● Centrify
● ForgeRock OpenAM
© 2013 Solers, Inc.
11

HIDS
●
●
Centralized Host-based Intrusion Detection System
(HIDS) solution
Ensures integrity of critical system and
configuration files across the infrastructure,
including:
●
●
●
●
●
Computing device firmware
Networking device firmware
Storage device firmware
Operating systems
Applications and services
●
Technologies Used
●
Tripwire Enterprise
© 2013 Solers, Inc.
12

Anti-Malware/HIPS
●
●
Provides virus scanning and Host-based Intrusion
Prevention System (HIPS) capabilities across all
machines and operating systems
Centralized virus signature and HIPS policy
management (automated deployments and
updates)
●
Technologies Used
●
McAfee VirusScan Enterprise, HIPS, and ePolicy Orchestrator
© 2013 Solers, Inc.
13

Network Management
●
●
●
Domain Name Service (DNS) Server
Dynamic Host Configuration Protocol (DHCP) Server
Network Time Protocol (NTP) Server
●
Technologies Used
●
●
●
Microsoft Windows DNS and Time Services (integrated with Active
Directory)
Red Hat DCHP Server
Red Hat NTP Server
© 2013 Solers, Inc.
14

Database
●
Highly Available Relational Database Solution
●
●
Two Oracle Database 11gR2 Enterprise Edition Database Server instances
●
●
One primary instance providing client access
One identical standby instance to receive/apply redo operations from
primary database
Oracle Data Guard configuration established between primary & standby
database servers to maintain duplicate copy of operational database
●
Supports high database availability and fast start failover
●
Technologies Used
● Oracle Database 11gR2 Enterprise Edition with Data Guard
● Hibernate (database client access)
© 2013 Solers, Inc.
15

Data Intake & Transmission
●
●
FTP, FTPS, and SFTP client and server solutions
Used to obtain product and ancillary data from
providers (intake), and deliver product and ancillary
data to consumers (transmission) via push or pull
●
Technologies Used
● Apache FtpServer (FTP and FTPS Server)
● Apache SSHD (SFTP Server)
● Apache Commons Library (FTP, FTPS, and SFTP Client)
© 2013 Solers, Inc.
16

Scheduling
●
Schedules periodic operations to be performed
within the infrastructure
●
●
●
Product and ancillary data inventory cleanup (expired files)
Subscription-specific product and ancillary data acquisition
Extensible to accommodate future scheduling needs
●
Technologies Used
●
Terracotta Quartz Scheduler
© 2013 Solers, Inc.
17

System Backup
●
●
Performs periodic backup of specific system data
and files to support on-site archive and recovery
Backups include:
●
●
●
●
VM image files
Database contents
Log files
Configuration files
●
Technologies Used
●
EMC NetWorker
© 2013 Solers, Inc.
18

ESPDS Scalable and Secure
Infrastructure Benefits
●
●
●
To End Users
●
●
Ensures highly available and reliable access to human and machine
interfaces that scales to accommodate the growing user and data
demands
Provides flexibility to quickly adapt to changes in end user
requirements
To System Operators/Administrators
●
●
●
Easily scalable hardware and software
Provides automated operations
Compliant with IT security requirements for a High Impact system
To NOAA/NESDIS As A Whole
●
●
●
Scalable and secure foundation to support enterprise environmental
satellite services across NOAA/NESDIS
Removes mission-specific stovepiping
Paving the path toward modernized data centers
© 2013 Solers, Inc.
19

Questions
© 2013 Solers, Inc.
20