WebAssembly (Wasm) and Rust: Dynamic Deployment of an OpenID Connect (OIDC) Filter to an Istio Service Mesh

Introduction

Organizations that run a large, distributed architecture are beginning to deploy Envoy on a large scale.

Originally built at Lyft, Envoy is an open source, edge and service proxy that abstracts the networking functionality away from applications, providing common, platform-agnostic features. Envoy proxies can be deployed beside your applications as a sidecar or run as an edge proxy.

What problem are we trying to solve by using Envoy in a service mesh? With all service traffic flowing through an Envoy mesh, it becomes possible to consistently control and observe what’s going on in your network. Envoy handles most network features– service discovery, access logging, metrics monitoring, tracing, authentication and authorization– configured via a control plane such as Istio, Google Traffic Director or AWS App Mesh. It pushes these concerns into the platform itself, so that developers can focus on the business logic of their applications, delivering services quickly and continuously using their choice of languages and technologies.

The Envoy model imposed a monolithic build process, and required extensions to be written in C++, limiting the developer ecosystem. Rolling out a new extension to the fleet required pushing new binaries and rolling restarts, which can be difficult to coordinate, and risk downtime. This also incentivized developers to upstream extensions into Envoy that were used by only a small percentage of deployments, just to piggyback on its release mechanisms.

Google has been working with the Envoy community to build Wasm extensibility into Envoy and contribute it upstream. It is now available as Alpha in the Envoy build shipped with Istio 1.5, with source in the envoy-wasm development fork and work is ongoing to merge it into the main Envoy tree. The implementation uses the WebAssembly runtime built into Google’s high performance V8 engine.

In addition to the underlying runtime, the Envoy project team also built:

  • A generic Application Binary Interface (ABI) for embedding Wasm in proxies, which means compiled extensions will work across different versions of Envoy - or even other proxies, should they choose to implement the ABI

  • SDKs for easy extension development in C++, Rust, and AssemblyScript, with more to inevitably follow

  • Comprehensive samples and instructions on how to deploy in Istio and standalone Envoy

  • Abstractions to allow for other Wasm runtimes to be used, including a ‘null’ runtime which simply compiles the extension natively into Envoy — very useful for testing and debugging

Using Wasm for extending Envoy brings in several significant benefits:

  • Agility: Extensions can be delivered and reloaded at runtime using the Istio control plane. This enables an accelerated develop → test → release cycle for extensions without requiring repetitive Envoy rollouts.

  • Stock Releases: Once merging into the main tree is complete, Istio and others will be able to use stock releases of Envoy, instead of custom builds. This will also free the Envoy community to move some of the built-in extensions to this model, thereby reducing their supported footprint.

  • Reliability and Isolation: Extensions are deployed inside a sandbox with resource constraints, which means they can now crash, or leak memory, without bringing the whole Envoy process down. CPU and memory usage can also be constrained.

  • Security: The sandbox has a clearly defined API for communicating with Envoy, so extensions only have access to, and can modify, a limited number of properties of a connection or request. Furthermore, because Envoy mediates this interaction, it can hide or sanitize sensitive information from the extension (e.g., Authorization and Cookie HTTP headers, or the client’s IP address).

  • Flexibility: over thirty (30) programming languages can be compiled to WebAssembly, allowing developers from all backgrounds - C++, Go, Rust, Java, TypeScript, and more to write Envoy extensions in their language of choice.

With Wasm support landing in Envoy, the extensibility of Envoy is assured and the extensibility of the Envoy-based istio-proxy is about to skyrocket to a new level.

The Wasm-based Envoy Filter Lifecycle

Refer to the figure below for a better understanding of the flow in a Wasm filter for Envoy. A custom WebAssembly (Wasm) code is represented by the yellow box. Envoy itself calls the code for lifecycle events like starting or configuring the plugin, or when headers or request bodies are to be processed.

The Wasm code can handle events, call functions in Envoy, and return a pass/fail to let Envoy know if it should continue processing the chain. Suppose the Wasm code needs to alter the HTTP request. The custom code in the yellow has access to the headers and can make changes to the request headers before the application or microservice ever sees it. Alternatively, the Wasm code may manipulate the response headers after the application (or microservice) container has set them. The Wasm code, for example, could request an HTTP call to check with an authorization (AuthZ) server. If the authorization server denies the access, the Wasm code would return a value to Envoy to fail the call, and block the request from reaching subsequent filters. This effectively blocks the traffic, since the user or service was calling .

Extending the Envoy Filter with Wasm - click image to enlarge

The UML model for an Envoy Filter is shown below. We use a number of these methods in our Rust-based Filter, as you will see below.

Envoy Filter UML Model - click image to enlarge

Deploying an OpenID Connect (OIDC) Filter into Istio to Protect Services

Credits

The major contributor to the Rust source code and use of Wasm is dgrimm@redhat.com.

Concept of Operations

Before we look at the details of our OIDC Filter, which is an Envoy-based Istio-Proxy, we are going to describe the Authorization Grant OAuth2 Flow. One of the many benefits of deploying applications to Istio is the availability of OAuth2 features built into it. In this case, our OIDC filter does not provide the JWT validation. We leave this up to Istio to provide that functionality.

Envoy (Istio-Proxy) HTTP Filter OAuth2 Flow - click image to enlarge

The flow for an unauthenticated user occurs as follows:

  1. The end user accesses a protected link via a React front-end in the user’s browser.
  2. The Envoy-based HTTP OIDC filter forces a redirect to an Authorization Server (OpenID Provider or OP).
  3. The user enters their credentials via a web form and logs in.
  4. The OIDC client (React application) receives an Authorization Code.
  5. The Authorization Code is presented to the application service that is front-ended by the OIDC Filter.
  6. The OIDC Filter validates the Authorization Code with the OP.
  7. The OP returns an Access Token (with and ID_Token) to the OIDC client.
  8. The React application access the application service with the JSON Web Token (JWT).
  9. Istio validates the JWT and the OIDC Filter writes the JWT to an http cookie and passes the request to the application service.
  10. The React front-end displays the requested data recieved in the JSON payload from the application service.

When the Wasm plugin (the Wasm binary that contains the filter) is loaded, a root context is created. The root context has the same lifetime as the VM instance, which executes our filter and is used for:

  • interactions at initial setup between your code and the Envoy Proxy
  • interactions that outlive a request

The function on_configure is only invoked in RootContext by the Envoy Proxy to pass in the VM and plugin configurations.

impl RootContext for OIDCRootContext {
    
    fn on_vm_start(&mut self, _vm_configuration_size: usize) -> bool {
        info!("VM STARTED");
        true
    }

    fn on_configure(&mut self, _plugin_configuration_size: usize) -> bool {
        info!("READING CONFIG");
        if self.config.auth_cluster == "" {
            info!("CONFIG EMPTY");
            if let Some(config_bytes) = self.get_configuration() {
                info!("GOT CONFIG");
                // TODO: some proper error handling here
                let cfg: Value = serde_json::from_slice(config_bytes.as_slice()).unwrap();
                self.config.auth_cluster = cfg.get("auth_cluster").unwrap().as_str().unwrap().to_string();
                self.config.auth_host = cfg.get("auth_host").unwrap().as_str().unwrap().to_string();
                self.config.login_uri = cfg.get("login_uri").unwrap().as_str().unwrap().to_string();
                self.config.token_uri = cfg.get("token_uri").unwrap().as_str().unwrap().to_string();
                self.config.client_id = cfg.get("client_id").unwrap().as_str().unwrap().to_string();
                self.config.client_secret = cfg.get("client_secret").unwrap().as_str().unwrap().to_string();
            }
        }
        true
    }

The network traffic handled by Envoy Proxy will flow through the filter chain associated with the listener that receives the traffic. For each new stream that flows through a filter chain, Envoy Proxy creates a new context which lasts until the stream ends.

The Context base class provides hooks (callbacks) in the form of onXXXX(…) virtual functions for HTTP and TCP traffic which are invoked as the Envoy Proxy iterates through the filter chain. Note that which callbacks are invoked on Context depends on the level of the filter chain your filter is inserted to. For example, the FilterHeadersStatus onRequestHeaders(uint32_t) is invoked only on WASM filters that are part of an HTTP-level filter chain and won’t be on TCP-level filters.

The implementation of Context base class is used by Envoy Proxy for interacting with the filter code throughout the lifespan of the stream. You can manipulate/mutate the traffic from within these callback functions. The SDK provides specific functions for manipulating HTTP request/response header (e.g. getRequestHeader, addRequestHeader, etc), HTTP body, TCP streams (e.g. getBufferBytes, setBufferBytes), etc. Each callback function returns a status through which you can tell Envoy Proxy whether or not to pass the processing of the stream to the next filter in the chain.

The OIDC HTTP filter, which is deployable to an Envoy proxy (istio-proxy) in an Istio Service Mesh, is designed to authenticate a user before he/she can access a service or microservice. We will show some logs of the istio-proxy that will clearly show the process flow inside of the Rust-based HTTP filter.

The sequence for end user authentication is well-documented by looking at the istio-proxy logs.

Each request directed to our httpbin service needs to be authorized by having an authorization header in the request. If it does, the JWT is then checked for validity by Istio. If the token is valid, the request is passed on to the service. Otherwise, we check for the presence of our cookie that contains the id_token. If no cookie is present, we will look for an authorization code to exchange for a token and set the cookie. If an authorization code is not in the request, we will force an authentication event to the XtremeCloud Single Sign-On (SSO) server, which is the OpenID Provider (OP).

2020-09-23T16:45:04.091190Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-23T16:45:04.092126Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-23T16:45:04.092199Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No cookie found. Checking for code in request parameters
2020-09-23T16:45:04.092219Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No code found. Redirecting to auth endpoint
2020-09-23T16:45:04.092803Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=12
2020-09-23T16:45:04.092862Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=6
2020-09-23T16:45:04.092879Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=10
2020-09-23T16:45:04.092891Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=14
2020-09-23T16:45:07.109494Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-23T16:45:07.109578Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-23T16:45:07.109858Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No cookie found. Checking for code in request parameters
2020-09-23T16:45:07.109905Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Code found. Dispatching HTTP call to token endpoint
2020-09-23T16:45:07.109918Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Sending data to token endpoint: grant_type=authorization_code&code=20e8d362-1997-4782-9819-92731ef90ad4.dc956c75-1852-4f48-ba44-9c3fdff554f5.test&redirect_uri=http%3A%2F%2Fhttpbin.eupraxialabs.com%2Fheaders&client_id=test&client_secret=daf72f94-2fd1-4241-9be6-532954b167fb
2020-09-23T16:45:07.131897Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Got response from token endpoint
2020-09-23T16:45:07.132297Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log oidc-filter oidc-filter_root: Received json blob: {"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJ4dThSZjBoVUxYYUNkN0RnSks4UTRzamN1TjV3aWNfeUpNUHIwMkxFUXlBIn0.eyJleHAiOjE2MDA4ODEzMDcsImlhdCI6MTYwMDg3OTUwNywiYXV0aF90aW1lIjoxNjAwODc5NTA3LCJqdGkiOiJlYWZmZWRhOC02ODFlLTQ2MzMtOTNkMS04NmNlNTM1ZmIyZjciLCJpc3MiOiJodHRwczovL3Nzby1kZXYuZXVwcmF4aWFsYWJzLmNvbS9hdXRoL3JlYWxtcy9tYXN0ZXIiLCJhdWQiOlsibWFzdGVyLXJlYWxtIiwiYWNjb3VudCJdLCJzdWIiOiJmOTUzNmYwMC00M2FmLTRiNWEtOGI1Ny0zNzk5OWI0YjU3MWIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJ0ZXN0Iiwic2Vzc2lvbl9zdGF0ZSI6ImRjOTU2Yzc1LTE4NTItNGY0OC1iYTQ0LTljM2ZkZmY1NTRmNSIsImFjciI6IjEiLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsiY3JlYXRlLXJlYWxtIiwib2ZmbGluZV9hY2Nlc3MiLCJhZG1pbiIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsibWFzdGVyLXJlYWxtIjp7InJvbGVzIjpbInZpZXctaWRlbnRpdHktcHJvdmlkZXJzIiwidmlldy1yZWFsbSIsIm1hbmFnZS1pZGVudGl0eS1wcm92aWRlcnMiLCJpbXBlcnNvbmF0aW9uIiwiY3JlYXRlLWNsaWVudCIsIm1hbmFnZS11c2VycyIsInF1ZXJ5LXJlYWxtcyIsInZpZXctYXV0aG9yaXphdGlvbiIsInF1ZXJ5LWNsaWVudHMiLCJxdWVyeS11c2VycyIsIm1hbmFnZS1ldmVudHMiLCJtYW5hZ2UtcmVhbG0iLCJ2aWV3LWV2ZW50cyIsInZpZXctdXNlcnMiLCJ2aWV3LWNsaWVudHMiLCJtYW5hZ2UtYXV0aG9yaXphdGlvbiIsIm1hbmFnZS1jbGllbnRzIiwicXVlcnktZ3JvdXBzIl19LCJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6Im9wZW5pZCBwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJhZG1pbiJ9.AJ0toWo6UjcrmKDDTlOjPxd55WEGRgmz6gOVXxd7VjWXkhlJ5JrevK7s0trR2zdhVNp4pcdDUS9tP6qdm2FeMdnVq8V8Z2eNCek9BU1G-xPMZ_0T4NyTI0EAZIOvjmhjFtb3RMQgwMeOGVu_ABK15JpLzA2PU1k-z6638GgtoJu30doZTiU8creSbOZYMvE4qi31SaHYhNqvGoyIsuoRH5Ovjrflz7IqF1T7Mxf0UyAkwvjyD69mSsoRgpy1HcN6s4ajEgmL8iq5YSszHu_0uZoeneFIU84CUqZwZyY7OlvrSgffymCvl67JI8JF1LRU10vNt0PWcnCwp2fwRsakxw","expires_in":1800,"id_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJ4dThSZjBoVUxYYUNkN0RnSks4UTRzamN1TjV3aWNfeUpNUHIwMkxFUXlBIn0.eyJleHAiOjE2MDA4ODEzMDcsImlhdCI6MTYwMDg3OTUwNywiYXV0aF90aW1lIjoxNjAwODc5NTA3LCJqdGkiOiI1NjRmNzkyYS1mZjQ3LTQ4OWMtYWMxYi0xMzk4ODM0ZTExYzYiLCJpc3MiOiJodHRwczovL3Nzby1kZXYuZXVwcmF4aWFsYWJzLmNvbS9hdXRoL3JlYWxtcy9tYXN0ZXIiLCJhdWQiOiJ0ZXN0Iiwic3ViIjoiZjk1MzZmMDAtNDNhZi00YjVhLThiNTctMzc5OTliNGI1NzFiIiwidHlwIjoiSUQiLCJhenAiOiJ0ZXN0Iiwic2Vzc2lvbl9zdGF0ZSI6ImRjOTU2Yzc1LTE4NTItNGY0OC1iYTQ0LTljM2ZkZmY1NTRmNSIsImFjciI6IjEiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsInByZWZlcnJlZF91c2VybmFtZSI6ImFkbWluIn0.P-j82N4wEkS2Pzj1VB8riMYbqq7mrDR1xYqWSOw-H6OyV-4J1fjH8AaEVOrrpwu2MLLhAGhrfhKEDYURkiSh-CbAozBRydovi4_S4j5p9rs26HckFnsSLd7yFKZdMLUqTZZAtCr__GUdp6iVoelOLInKhpfgKUB6n35fgSj0Y3vz0TPLMTwyIPyTeqiugRyqAcBfzjvxw47Tta7Jp6MF5mrVVQkPc7w60-rAsDYD1c6vbE8omYFgwIXsNUPBWkGY86cb7wI2vVY9Hc24_x75fzeMl6Gz7UWXZqg0MJWkvFEeba8lh3oyAvtyLtfW5IDN2srUgGnGTr4g7AAqCC6o7w","not-before-policy":0,"refresh_expires_in":1800,"refresh_token":"eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICI2ZWY0ODU0ZC05NWZjLTQ2MjktYTBhMy05ZGMwZTNiNGVjMjkifQ.eyJleHAiOjE2MDA4ODEzMDcsImlhdCI6MTYwMDg3OTUwNywianRpIjoiYTBlOWVmNzktMWM5ZS00OTJjLWFkZDAtM2FkNGE1Mzc4NzU5IiwiaXNzIjoiaHR0cHM6Ly9zc28tZGV2LmV1cHJheGlhbGFicy5jb20vYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoiaHR0cHM6Ly9zc28tZGV2LmV1cHJheGlhbGFicy5jb20vYXV0aC9yZWFsbXMvbWFzdGVyIiwic3ViIjoiZjk1MzZmMDAtNDNhZi00YjVhLThiNTctMzc5OTliNGI1NzFiIiwidHlwIjoiUmVmcmVzaCIsImF6cCI6InRlc3QiLCJzZXNzaW9uX3N0YXRlIjoiZGM5NTZjNzUtMTg1Mi00ZjQ4LWJhNDQtOWMzZmRmZjU1NGY1Iiwic2NvcGUiOiJvcGVuaWQgcHJvZmlsZSBlbWFpbCJ9.XRYcv9dp7aAlKWpv6iFIocFcC7T_BnHFocfB9tHb2fA","scope":"openid profile email","session_state":"dc956c75-1852-4f48-ba44-9c3fdff554f5","token_type":"bearer"}
2020-09-23T16:45:07.132385Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: id_token found. Setting cookie and redirecting...

Building of the HTTP Filter

use log::{debug, info};
use proxy_wasm::traits::*;
use proxy_wasm::types::*;
use url::form_urlencoded;
use serde_json::{Value};
use std::time::Duration;

//use no_mangle to preserve the function name and use _start, which is a special function.
#[no_mangle]
pub fn _start() {
    proxy_wasm::set_log_level(LogLevel::Trace);
    proxy_wasm::set_root_context(|_| -> Box<dyn RootContext> {
        Box::new(OIDCRootContext{
            config: FilterConfig{
                auth_cluster: "".to_string(),
                auth_host: "".to_string(),
                login_uri: "".to_string(),
                token_uri: "".to_string(),
                client_id: "".to_string(),
                client_secret: "".to_string(),
            }
        })
    });
}

struct OIDCFilter {
    authority: String,
    path: String,
    config: FilterConfig,
}

struct OIDCRootContext {
    config: FilterConfig
}

struct FilterConfig {
    auth_cluster: String,
    auth_host: String,
    login_uri: String,
    token_uri: String,
    client_id: String,
    client_secret: String,
}

impl OIDCFilter {
    fn is_authorized(&self) -> bool {
        let headers = self.get_http_request_headers();
        for (key,_value) in headers.iter() {
            if key.to_lowercase().trim() == "authorization" {
                return true;
            }
        }
        return false;
    }

    fn get_code(&self) -> String {
        let path = self.get_http_request_header(":path").unwrap();
        let path_parts: Vec<_> = path.split("?").collect();
        if path_parts.len() < 2 {
            return "".to_string()
        }
        let query = path_parts[1].to_string();
        let encoded = form_urlencoded::parse(query.as_bytes());
        for (k, v) in encoded {
            if k == "code" {
                return v.to_owned().to_string();
            }
        }
        return "".to_string();
    }

    fn get_cookie(&self, name: &str) -> String {
        let headers = self.get_http_request_headers();
        let mut cookies = "";
        for (key,value) in headers.iter() {
            if key.to_lowercase().trim() == "cookie" {
                cookies = value;
            }
        }
        let assignments: Vec<_> = cookies.split(";").collect();
        for assignment in assignments {
            let kvpair: Vec<_> = assignment.split("=").collect();
            if kvpair[0] == name {
                return kvpair[1].to_owned();
            }
        }
        return "".to_owned()
    }

    fn get_redirect_uri(&self, current_uri: &str) -> String {
        let encoded: String = form_urlencoded::Serializer::new(String::new())
            .append_pair("client_id", "test")
            .append_pair("response_type", "code")
            .append_pair("scope", "openid profile email")
            .append_pair("redirect_uri", current_uri)
            .finish();
        
        format!("{}?{}", self.config.login_uri, encoded)
    }

    fn get_http_authority(&self) -> String {
        return self.authority.to_owned();
    }

    fn set_http_authority(&mut self, authority: String) {
        self.authority = authority.to_owned();
    }

    fn get_http_path(&self) -> String {
        return self.path.to_owned();
    }

    fn set_http_path(&mut self, path: String) {
        self.path = path.to_owned();
    }

}

impl HttpContext for OIDCFilter {

    fn on_http_request_headers(&mut self, _: usize) -> Action {
        let host = self.get_http_request_header(":authority").unwrap();
        let path = self.get_http_request_header(":path").unwrap();
        self.set_http_authority(host.to_owned());
        
        // TODO: move this into its own fn filter_query
        let path_parts: Vec<_> = path.split("?").collect();
        let mut redirect_path_serializer: url::form_urlencoded::Serializer<String> = form_urlencoded::Serializer::new(String::new());
        let redirect_path: String;
        if path_parts.len() < 2 {
            redirect_path = path.clone();
        } else {
            for (key, value) in form_urlencoded::parse(path_parts[1].as_bytes()).into_owned() {
                if key != "code" && key != "session_state" {
                    redirect_path_serializer.append_pair(key.as_str(), value.as_str());
                }
            }
            let query: String = redirect_path_serializer.finish();
            if query == "" {
                redirect_path = path_parts[0].to_string();
            } else {
                redirect_path = format!("{}?{}", path_parts[0], query);    
            }
        }
        self.set_http_path(redirect_path.to_owned());

        info!("Request received");
        if !self.is_authorized() {
            info!("No auth header present. Checking for cookie containing id_token");
            let token = self.get_cookie("oidcToken");
            if token != "" {
                info!("Cookie found, setting auth header");
                self.set_http_request_header("Authorization", Some(&format!("Bearer {}", token)));
                return Action::Continue
            }

            info!("No cookie found. Checking for code in request parameters");
            let code = self.get_code();
            if code != "" {
                info!("Code found. Dispatching HTTP call to token endpoint");
                let data: String = form_urlencoded::Serializer::new(String::new())
                    .append_pair("grant_type", "authorization_code")
                    .append_pair("code", code.as_str())
                    .append_pair("redirect_uri", format!("http://{}{}", host, redirect_path).as_str())
                    .append_pair("client_id", self.config.client_id.as_str())
                    .append_pair("client_secret", self.config.client_secret.as_str())
                    .finish();
                info!("Sending data to token endpoint: {}", data);
                
                self.dispatch_http_call(
                    self.config.auth_cluster.as_str(), vec![
                        (":method", "POST"),
                        (":path", self.config.token_uri.as_str()),
                        (":authority", self.config.auth_host.as_str()),
                        ("Content-Type", "application/x-www-form-urlencoded"),
                    ],
                    Some(data.as_bytes()),
                    vec![],
                    Duration::from_secs(5)
                ).unwrap();
                return Action::Pause
            }

            info!("No code found. Redirecting to auth endpoint");
            self.send_http_response(
                302,
                vec![("Location", self.get_redirect_uri(format!("http://{}{}", host, path).as_str()).as_str())],
                Some(b""),
            );
            return Action::Pause
        }
        Action::Continue
    }
}

impl Context for OIDCFilter {

    fn on_http_call_response(&mut self, _: u32, _: usize, body_size: usize, _: usize) {
        info!("Got response from token endpoint");
        let host = self.get_http_authority();
        let path = self.get_http_path();
        if let Some(body) = self.get_http_call_response_body(0, body_size) {
            let data: Value = serde_json::from_slice(body.as_slice()).unwrap();
            debug!("Received json blob: {}", data);
            if data.get("error") != None {
                info!("Error fetching token: {}, {}", data.get("error").unwrap(), data.get("error_description").unwrap());
                return
            }
            if data.get("id_token") != None {
                info!("id_token found. Setting cookie and redirecting...");
                self.send_http_response(
                    302,
                    vec![
                        ("Set-Cookie", format!("oidcToken={};Max-Age={}", data.get("id_token").unwrap(), data.get("expires_in").unwrap()).as_str()),
                        ("Location", format!("http://{}{}", host, path).as_str()),
                    ],
                    Some(b""),
                );
                return
            }
        }
    }
}

impl Context for OIDCRootContext {}

impl RootContext for OIDCRootContext {
    
    fn on_vm_start(&mut self, _vm_configuration_size: usize) -> bool {
        info!("VM STARTED");
        true
    }

    fn on_configure(&mut self, _plugin_configuration_size: usize) -> bool {
        info!("READING CONFIG");
        if self.config.auth_cluster == "" {
            info!("CONFIG EMPTY");
            if let Some(config_bytes) = self.get_configuration() {
                info!("GOT CONFIG");
                // TODO: some proper error handling here
                let cfg: Value = serde_json::from_slice(config_bytes.as_slice()).unwrap();
                self.config.auth_cluster = cfg.get("auth_cluster").unwrap().as_str().unwrap().to_string();
                self.config.auth_host = cfg.get("auth_host").unwrap().as_str().unwrap().to_string();
                self.config.login_uri = cfg.get("login_uri").unwrap().as_str().unwrap().to_string();
                self.config.token_uri = cfg.get("token_uri").unwrap().as_str().unwrap().to_string();
                self.config.client_id = cfg.get("client_id").unwrap().as_str().unwrap().to_string();
                self.config.client_secret = cfg.get("client_secret").unwrap().as_str().unwrap().to_string();
            }
        }
        true
    }

    fn create_http_context(&self, _context_id: u32, _root_context_id: u32) -> Box<dyn HttpContext> {
        Box::new(OIDCFilter{
            authority: "".to_string(),
            path: "".to_string(),
            config: FilterConfig{
                auth_cluster: self.config.auth_cluster.clone(),
                auth_host: self.config.auth_host.clone(),
                login_uri: self.config.login_uri.clone(),
                token_uri: self.config.token_uri.clone(),
                client_id: self.config.client_id.clone(),
                client_secret: self.config.client_secret.clone(),
            },
        })
    
    }

    fn get_type(&self) -> ContextType {
        ContextType::HttpContext
    }

}

Envoy Proxy Wasm SDK

Envoy Proxy runs Wasm filters inside a stack-based virtual machine, thus the filter’s memory is isolated from the host environment. All interactions between the embedding host (Envoy Proxy) and the Wasm filter are realized through functions and callbacks provided by the Envoy Proxy Wasm SDK. The Envoy Proxy WASM SDK has implementations in various programming languages like:

  • C++
  • Rust
  • AssemblyScript
  • Go - still experimental

At a high level, we needed to do three (3) things:

  • Build our Rust code, which implements and uses the ABI
  • Compile the code to WebAssembly
  • Deploy this into an Istio-based Envoy proxy and demonstrate its functionality.

First, we defined a special function called _start which is part of the ABI (we use the no_mangle macro to preserve the name) and lets us initialize things. In it, we set the log level to trace and register a HttpContext defined later. HTTPcontext is one of the three context types available, used to build HTTP filters, along with RootContext and StreamContext, which you can use for configuration and working with timers, and TCP filters, respectively. You can read the available APIs, they are fairly straightforward.

Create the makefile for our Rust-based Wasm filter:

build: oidc.wasm

oidc.wasm:
	cargo build --target wasm32-unknown-unknown --release
	cp target/wasm32-unknown-unknown/release/oidc_filter.wasm ./oidc.wasm

.PHONY: clean
clean:
	rm oidc.wasm || true

Build the Wasm filter by running make in the root of the provided repository:

$ make build

Let’s hit the httpbin application in Istio. You will be prompted to authenticate.

OpenID Provider (OP) Prompts for User Credentials - click image to enlarge

Following authentication, the request is sent to the httpbin service (the Relying Party (RP)) in Istio.

httpbin Service Protected by the OIDC Filter - click image to enlarge

Optimizing our Wasm Binary

By setting the lto parameter to true in the Cargo.toml ([profile-release] stanza) file, the Wasm image shrank from 1.7M to 370k and it runs faster.

The smaller image gives you the option of deploying it via a Kubernetes configmap or using an Init container. The larger-sized Wasm image can only be deployed via an Init container. We provide the options, to our subscriber, for a ConfigMap or an Init container in the provided Helm Chart.

[package]
name = "oidc-filter"
version = "0.9.0"
authors = ["Developer 1 <developer1@eupraxialabs.com>"]
edition = "2018"

[dependencies]
# Have to use a Eupraxia Labs fork because config reading is broken upstream
proxy-wasm = { git = "https://github.com/eupraxialabs/proxy-wasm-rust-sdk.git" }
#proxy-wasm = "0.1.2"
log = "0.4.8"
url = "2.1.1"
serde_json = "1.0"
state = "0.4.1"

[lib]
# crate-type
crate-type = ["cdylib"]
path = "src/lib.rs"

[profile.release]
lto = true             <==================== SHRINK YOUR CODE SIZE

Advanced Debugging

Let’s enable a Wasm filter to log at the DEBUG log-level when processing traffic which targets the httpbin service:

$ kubectl port-forward deployment/httpbin 15000

In a separate terminal:

$ curl -X POST "localhost:15000/logging?wasm=debug"
active loggers:
  admin: warning
  aws: warning
  assert: warning
  backtrace: warning
  cache_filter: warning
  client: warning
  config: warning
  connection: warning
  conn_handler: warning
  decompression: warning
  dubbo: warning
  file: warning
  filter: warning
  forward_proxy: warning
  grpc: warning
  hc: warning
  health_checker: warning
  http: warning
  http2: warning
  hystrix: warning
  init: warning
  io: warning
  jwt: warning
  kafka: warning
  lua: warning
  main: warning
  misc: warning
  mongo: warning
  quic: warning
  quic_stream: warning
  pool: warning
  rbac: warning
  redis: warning
  router: warning
  runtime: warning
  stats: warning
  secret: warning
  tap: warning
  testing: warning
  thrift: warning
  tracing: warning
  upstream: warning
  udp: warning
  wasm: debug

A review of the istio-proxy sidecar logs shows us some very useful debug information:

2020-09-21T19:56:25.361084Z	warning	envoy misc	[external/envoy/source/common/protobuf/utility.cc:198] Using deprecated option 'envoy.api.v2.Listener.use_original_dst' from file listener.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details.
2020-09-21T20:12:04.091735Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-21T20:12:04.091805Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-21T20:12:04.091828Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No cookie found. Checking for code in request parameters
2020-09-21T20:12:04.091834Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No code found. Redirecting to auth endpoint
2020-09-21T20:12:04.091954Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=12
2020-09-21T20:12:04.091994Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=6
2020-09-21T20:12:04.092001Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=10
2020-09-21T20:12:04.092006Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=14
2020-09-21T20:12:04.350764Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-21T20:12:04.350852Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-21T20:12:04.350878Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No cookie found. Checking for code in request parameters
2020-09-21T20:12:04.350955Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Code found. Dispatching HTTP call to token endpoint
2020-09-21T20:12:04.351093Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Sending data to token endpoint: grant_type=authorization_code&code=d049176a-5f17-4faa-930b-aacfe19f6724.9066f319-5f95-4988-9597-f83416154b19.test&redirect_uri=http%3A%2F%2Fhttpbin.eupraxialabs.com%2Fheaders&client_id=test&client_secret=daf72f94-2fd1-4241-9be6-532954b167fb
2020-09-21T20:12:04.369054Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Got response from token endpoint
2020-09-21T20:12:04.369179Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log oidc-filter oidc-filter_root: Received json blob: {"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJ4dThSZjBoVUxYYUNkN0RnSks4UTRzamN1TjV3aWNfeUpNUHIwMkxFUXlBIn0.eyJleHAiOjE2MDA3MTkxODQsImlhdCI6MTYwMDcxOTEyNCwiYXV0aF90aW1lIjoxNjAwNzE2OTA0LCJqdGkiOiJkY2VjOTM0My03OWY2LTQ0YzYtYWRjMC0zZGZhYjA3NDU0Y2EiLCJpc3MiOiJodHRwczovL3Nzby1kZXYuZXVwcmF4aWFsYWJzLmNvbS9hdXRoL3JlYWxtcy9tYXN0ZXIiLCJhdWQiOlsibWFzdGVyLXJlYWxtIiwiYWNjb3VudCJdLCJzdWIiOiJmOTUzNmYwMC00M2FmLTRiNWEtOGI1Ny0zNzk5OWI0YjU3MWIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJ0ZXN0Iiwic2Vzc2lvbl9zdGF0ZSI6IjkwNjZmMzE5LTVmOTUtNDk4OC05NTk3LWY4MzQxNjE1NGIxOSIsImFjciI6IjAiLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsiY3JlYXRlLXJlYWxtIiwib2ZmbGluZV9hY2Nlc3MiLCJhZG1pbiIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsibWFzdGVyLXJlYWxtIjp7InJvbGVzIjpbInZpZXctaWRlbnRpdHktcHJvdmlkZXJzIiwidmlldy1yZWFsbSIsIm1hbmFnZS1pZGVudGl0eS1wcm92aWRlcnMiLCJpbXBlcnNvbmF0aW9uIiwiY3JlYXRlLWNsaWVudCIsIm1hbmFnZS11c2VycyIsInF1ZXJ5LXJlYWxtcyIsInZpZXctYXV0aG9yaXphdGlvbiIsInF1ZXJ5LWNsaWVudHMiLCJxdWVyeS11c2VycyIsIm1hbmFnZS1ldmVudHMiLCJtYW5hZ2UtcmVhbG0iLCJ2aWV3LWV2ZW50cyIsInZpZXctdXNlcnMiLCJ2aWV3LWNsaWVudHMiLCJtYW5hZ2UtYXV0aG9yaXphdGlvbiIsIm1hbmFnZS1jbGllbnRzIiwicXVlcnktZ3JvdXBzIl19LCJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6Im9wZW5pZCBwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJhZG1pbiJ9.CQmJ0pZSdnktKbH-mPvt9L92Pklfv0Au_jKsPclVKihGlHegQc_xvgifBpJ8bXWsn1gzUkIaheU3QiI2nQr2uwE6kc_xtsrjqXYlvPHEQ65oLSlwkbbFq8NFsDYfIwk000TRap6WCLeWdsIPfDBIFDTEIRme6d227dm1g9j37IfHnth57bFa-qm4rNC-dOIhDbtpe5aO-Y3GLYWUjTyJCGR68gESdeOFqUdi-qagc-3mMlEYWWPvnyEKeaOBGumIkSKfkxt10b-Zj8_45k2FWOjNV4V2Qx1i6hELXC7P4c20T53srfJNXHOtq7Tn4M1BZZ9eTYhdMDnoFgNth3aSig","expires_in":60,"id_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJ4dThSZjBoVUxYYUNkN0RnSks4UTRzamN1TjV3aWNfeUpNUHIwMkxFUXlBIn0.eyJleHAiOjE2MDA3MTkxODQsImlhdCI6MTYwMDcxOTEyNCwiYXV0aF90aW1lIjoxNjAwNzE2OTA0LCJqdGkiOiJmMmZlZWZkNC02MTU2LTRiNzUtODkxNy05MDA1N2M3NGZlYjMiLCJpc3MiOiJodHRwczovL3Nzby1kZXYuZXVwcmF4aWFsYWJzLmNvbS9hdXRoL3JlYWxtcy9tYXN0ZXIiLCJhdWQiOiJ0ZXN0Iiwic3ViIjoiZjk1MzZmMDAtNDNhZi00YjVhLThiNTctMzc5OTliNGI1NzFiIiwidHlwIjoiSUQiLCJhenAiOiJ0ZXN0Iiwic2Vzc2lvbl9zdGF0ZSI6IjkwNjZmMzE5LTVmOTUtNDk4OC05NTk3LWY4MzQxNjE1NGIxOSIsImFjciI6IjAiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsInByZWZlcnJlZF91c2VybmFtZSI6ImFkbWluIn0.MJUxFl_YcEcuwas0VtZX-8bnzFSJVgWADuECcqqNd_lu3ttWdDcGHLFun59QwtkomC4OuEo4lrCVv1TawXaHMpwyEIi1NyooLl9Nu7j5Om1vNZUi1qOd2T5Z1fPjyVZjuce3CBT8ZXuS0mCs6Ch-AbVXE7WkEgA6xodbgiwtB7tgyK38Qt_AiEviAAVMo6lOSgTUAXFYYm1iRTrlpcidXlPWSdW990HQQcHDKS-7OaFtqFc_YbEt5lB7xZ6vYP_qflQS4J4OV8QXpMkI3fP4f7a6gdrvJvp3HiwuI90iO2wttEL48MnjqFdaJhkPisa5PdNthRvnPW66eL_54-NoPQ","not-before-policy":0,"refresh_expires_in":1800,"refresh_token":"eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICI2ZWY0ODU0ZC05NWZjLTQ2MjktYTBhMy05ZGMwZTNiNGVjMjkifQ.eyJleHAiOjE2MDA3MjA5MjQsImlhdCI6MTYwMDcxOTEyNCwianRpIjoiYTgxYTliMjktZmQxMy00NmJkLWI1MzQtYzcxNTk0NDNkZTQ1IiwiaXNzIjoiaHR0cHM6Ly9zc28tZGV2LmV1cHJheGlhbGFicy5jb20vYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoiaHR0cHM6Ly9zc28tZGV2LmV1cHJheGlhbGFicy5jb20vYXV0aC9yZWFsbXMvbWFzdGVyIiwic3ViIjoiZjk1MzZmMDAtNDNhZi00YjVhLThiNTctMzc5OTliNGI1NzFiIiwidHlwIjoiUmVmcmVzaCIsImF6cCI6InRlc3QiLCJzZXNzaW9uX3N0YXRlIjoiOTA2NmYzMTktNWY5NS00OTg4LTk1OTctZjgzNDE2MTU0YjE5Iiwic2NvcGUiOiJvcGVuaWQgcHJvZmlsZSBlbWFpbCJ9.QESzVqUQ5IywBZIvxu5ZoJauLuhFh_sZ4970AxAzxuA","scope":"openid profile email","session_state":"9066f319-5f95-4988-9597-f83416154b19","token_type":"bearer"}
2020-09-21T20:12:04.369189Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: id_token found. Setting cookie and redirecting...
2020-09-21T20:12:04.369316Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=12
2020-09-21T20:12:04.369327Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=6
2020-09-21T20:12:04.369333Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=10
2020-09-21T20:12:04.369337Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=14
2020-09-21T20:12:04.379803Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-21T20:12:04.379911Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-21T20:12:04.379970Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Cookie found, setting auth header
2020-09-21T20:12:04.384520Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=16
2020-09-21T20:12:04.384571Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=18
2020-09-21T20:12:04.384578Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=22
2020-09-21T20:12:04.384583Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=26
2020-09-21T20:12:05.958607Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-21T20:12:05.958679Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-21T20:12:05.958710Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Cookie found, setting auth header
2020-09-21T20:12:05.961994Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=16
2020-09-21T20:12:05.962052Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=18
2020-09-21T20:12:05.962062Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=22
2020-09-21T20:12:05.962068Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=26
2020-09-21T20:12:06.372882Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-21T20:12:06.372954Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-21T20:12:06.372986Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Cookie found, setting auth header
2020-09-21T20:12:06.381229Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=16
2020-09-21T20:12:06.381281Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=18
2020-09-21T20:12:06.381384Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=22
2020-09-21T20:12:06.381405Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=26
2020-09-21T20:12:06.753037Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Request received
2020-09-21T20:12:06.753111Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: No auth header present. Checking for cookie containing id_token
2020-09-21T20:12:06.753142Z	info	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1107] wasm log oidc-filter oidc-filter_root: Cookie found, setting auth header
2020-09-21T20:12:06.757917Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=16
2020-09-21T20:12:06.757967Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=18
2020-09-21T20:12:06.757974Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=22
2020-09-21T20:12:06.757978Z	debug	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1104] wasm log stats_inbound: [extensions/stats/plugin.cc:612]::report() metricKey cache hit , stat=26
2020-09-21T20:24:01.844038Z	warning	envoy config	[bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:92] StreamAggregatedResources gRPC config stream closed: 13, 
2020-09-21T20:24:02.297652Z	warning	envoy misc	[external/envoy/source/common/protobuf/utility.cc:198] Using deprecated option 'envoy.api.v2.Listener.use_original_dst' from file listener.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details.
2020-09-21T20:24:05.004681Z	info	Subchannel Connectivity change to CONNECTING
2020-09-21T20:24:05.004822Z	info	transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2020-09-21T20:24:05.004991Z	info	pickfirstBalancer: HandleSubConnStateChange: 0xc000c26090, {CONNECTING <nil>}
2020-09-21T20:24:05.005057Z	info	Channel Connectivity change to CONNECTING
2020-09-21T20:24:05.005061Z	info	Subchannel picks a new address "istiod.istio-system.svc:15012" to connect
2020-09-21T20:24:05.012013Z	info	Subchannel Connectivity change to READY
2020-09-21T20:24:05.012057Z	info	pickfirstBalancer: HandleSubConnStateChange: 0xc000c26090, {READY <nil>}
2020-09-21T20:24:05.012064Z	info	Channel Connectivity change to READY

Trace Transactions with Jaeger

Let’s generate some traffic:

In the namespace where httpbin is deployed.

Davids-MacBook-Pro:example davidjbrewer$ kubectl port-forward deployment/httpbin 15000
Forwarding from 127.0.0.1:15000 -> 15000
Forwarding from [::1]:15000 -> 15000

$ while true; do
  curl -s http://httpbin.eupraxialabs.com > /dev/null
  echo -n .;
  sleep 0.2
done
Davids-MacBook-Pro:example davidjbrewer$ while true; do
>   curl -s http://httpbin.eupraxialabs.com > /dev/null
>   echo -n .;
>   sleep 0.2
> done
...........................

Look at the Jaeger dashboard by running:

Davids-MacBook-Pro:xtremecloud-sso-minikube davidjbrewer$ istioctl dashboard jaeger
http://localhost:50723

We can look at a trace in the Jaeger GUI.

Jaeger Trace through the Istio Ingressgateway - click image to enlarge

Summary

WebAssembly (Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications.