Service-Oriented Architecture

Overview

Service-Oriented Architecture (SOA) is a method for systems integration where operations within a given system are expressed outwardly as services which can then be called by other services and applications. These services can then be federated and orchestrated together into ad-hoc applications to fit a business need. A central concept of SOA is that any system or programming language can be used to create or consume any service using a standard set of protocols. Frequently, an SOA implementation also involves a generic messaging system implemented as a service (such as an Enterprise Message Bus), an orchestration framework, and a discovery process that collects and publishes metadata about available services.

When developing new services and upgrading existing services, the potential to leverage SOA will be considered when choosing products or design. Building to this model allows services to become more modular. An IT Services product may contain application components from multiple vendors or become a hybrid where portions of the service may be hosted locally and portions at other institutions or in the cloud.

IT Services will focus on improving the interoperability between the two service languages (Web Services and remctl) that are presently in common use at Stanford, make them both more operating-system agnostic, and improve the available toolkits for each.

Current State

  • Remctl/Unix: Stanford has developed a Service Oriented Architecture throughout most of the UNIX-based main infrastructure offerings based on remctl, a locally developed open-source product. Remctl is an extremely simple remote procedure call layer using Kerberos GSSAPI (Generic Security Service Application Program Interface) authentication that can be used to express functionality at the service layer. In practice, it is how the UNIX team does "programming in the large" and is central to Stanford's account provisioning and secure credential management processes, among others. It is also used as a back end for monitoring and administration actions on most of the main university services. There is a remctl client in use on Windows for service-level calls to UNIX, but as yet there is no native Windows remctl server. Remctl client bindings are available for C, Perl, Python, and PHP, and there is a separate native client and server implementation of the remctl protocol in Java, which could also be used on Windows.
  • Windows: SOAP (Simple Object Access Protocol) and WSDL (Web Service Definition Language) are the primary mechanisms for service-level communications to and between applications deployed on IT Services' Windows systems. Microsoft has invested heavily around Web Services and most Windows systems and applications export service-level functionality using SOAP or REST (Representational State Transfer)-based interfaces. Microsoft provides the Windows Communications Framework (WCF) API, which provides a simplified programming model for creating web services-enabled applications using either standard.
  • Applications/Services: Several IT Services' services applications and appliances provide SOAP interfaces, but the use of those interfaces is not yet widespread.
  • Event Model: IT Services implements a hub-and-spoke model for most service-based processes where events are pushed outwards from the interior. However, there are some places where there is a more complex messaging model in place. IT Services has found that most of the cases where this is necessary are representations of university data about people or accounts. For some communications with Administrative Systems, the SonicMQ service bus is being used, run out of the Administrative Systems middleware group. This system has performance and stability issues on the client side and is neither language-agnostic nor uses standard protocols. For communications within IT Services regarding this data, directory replication queries are used as an event stream for changes in person and account data. This has been implemented on both UNIX and Windows.
  • Issues: Major obstacles to implementation using an XML (eXtensible Markup Language)-based standard for service description on Unix include the complexity of schemas, the need to migrate into a more strongly typed protocol (remctl is very open-ended and therefore has uses for both human-to-service and service-to-service communications), and the need to run an HTTP (HyperText Transfer Protocol) daemon on all hosts.
  • Service Discovery: IT Services does not currently have a service discovery or service registry implementation.

Vision

At this time, IT Services is not pursuing a full-scale SOA framework implementation to encompass all possible services. SOA suites presently available from vendors represent a huge expense in transitioning from existing technology that dwarfs the expense involved in acquisition and installation. Any future all-encompassing SOA project should be taken on as a joint venture between IT Services and Administrative Systems and should encompass both computing infrastructure and university business services. Without that level of effort and buy-in, IT Services is unlikely to recoup its investment in the installation and systems integration of an IT Services-only solution.

IT Services will pursue a more conservative approach, emphasizing incremental improvements to existing systems, some service integrations with workflow technologies, and investigation into other technologies. A number of deployed services and services that will be coming into production use within the next year provide some components of SOA that will be leveraged to provide enhanced interaction between those services.

Technology trends that IT Services is tracking:

  • Protocols: There are two major protocols that are competing for the hearts and minds of SOA implementations: REST and SOAP. SOAP is based on an XML message format that can be transferred between applications using any Remote Procedure Call (RPC) mechanism, though HTTP and SMTP (Simple Mail Transfer Protocol) are the most common. REST is based on the concept of resources, represented by URIs (Uniform Resource Identifiers), and messages, which are typically carried over HTTP. SOAP and REST have been heavily deployed by Microsoft. REST has also been embraced by the emerging Ruby community as well as several major third-party cloud service providers (Google, Amazon, etc.). Both are likely to remain in widespread use for the foreseeable future.
  • Authentication: Neither REST nor SOAP has a particularly compelling authentication story yet at Stanford. SOAP uses HTTP essentially as a mute transport layer and encodes authentication in the posted XML documents. REST uses more features of the HTTP protocol, but that comes along with relying on authentication available via the web server. Typical authentication methods include:
    • Negotiate: The Negotiate-Auth solutions have had interoperability problems and are not always mutually authenticated.
    • X.509 certificates: Implementation of KX509 (see the Authentication strategy) is how IT Services will proceed with mutual, X.509-based authentication. Having that technology available will affect future evaluations and decisions in the SOA space.
    • SAML (Security Assertion Markup Language): Assertion or claims-based authentication is quickly becoming more prevalent as cloud services vendors provide federation between on premises authentication systems and their service offerings.
  • Authorization: Role-based authorization is quickly becoming the preferred way to control access to web services due to better compatibility with federated authentication scenarios. OAuth is gaining wide adoption (Google, Twitter, Facebook, etc.), especially for REST web services, to allow scenarios where the end user can delegate authorized access between multiple services with separate authentication systems.
  • Metadata: Behind the service layer, there are additional protocols for discovering and communicating metadata about services (WSDL is used by SOAP). These are generally also XML based.

Goals

  • Complete a remctl-based SOA across the entire UNIX infrastructure.
  • Bridge the gap between UNIX (remctl) and Windows (Web Services) infrastructures for service-level actions.
  • Consolidate registry-driven services to use the OpenLDAP (Lightweight Directory Access Protocol) directory and replication-based events.
  • Build a service discovery mechanism that can consume both WSDL and remctl help output.
  • Investigate and evaluate IT Services use cases for workflows and/or a messaging service.
  • Communicate with Administrative Systems applications over web services protocols for information about people, accounts, groups, and organizations. Migrate existing and future IT Services applications either to these web services interfaces or to IT Services-provided directory services (OpenLDAP or Active Directory).
  • Be able to engage with third-party vendors' service interfaces as required.

Roadmap

  • Implement ADFS (Active Directory Federation Service) v2 for SAML authentication interoperability.
  • Build libraries for SOAP/REST on UNIX so that a common toolkit for accessing services exported over SOAP or REST can be leveraged.
  • Export WSDL's for SOAP services running on UNIX.
  • Evaluate potential utilization of exported SOAP interfaces available from existing IT Services applications and data sources.
  • Work with Administrative Systems to design web services interfaces to institutional data based on internal and community needs.
  • Implement 'help' spec for remctl, allowing service discovery per system.
  • Implement registry of remctl servers, allowing service discovery across the infrastructure. Register SOAP services into that registry by consuming WSDL documents.
  • Implement a remctl server and a better remctl client on Windows, probably using .NET.
  • Continue to work with peer institutions to evolve the remctl protocol, implementation, and language support and build the growing community around the remctl protocol.
  • Continue to evaluate service buses or queuing systems as a possible solution for exchanging service information.
  • Evaluate the use of SAML and OAuth for leveraging external service providers.
  • Implement additional application registry harvesters to trigger based on directory replication data (based on current difficulties with Registry SonicMQ system).
  • Consolidate multiple technical approaches to SOA into a unified framework, if not a unified technology.

Measures of success

  • Complete rollout of remctl across all UNIX services.
  • All major infrastructure pieces expose functionality as SOA services.
  • IT Services offerings based on SOA.
  • All major UNIX infrastructure components implementing discovery via remctl 'help' functionality.
  • Number of integrations with third parties over a service framework (goal: all cloud services accessed via service calls).