OpenSSL Objects review: X509 certificate store

In the past year or so, while working on http://minerva.sandelman.ca, I’ve been through the less popular backrooms of the OpenSSL source code. The documentation for OpenSSL is extensive, but often not oriented towards discovery, but rather towards reference. As such, it’s generally quite good if you know what function you want, and just aren’t sure what one argument does. It doesn’t help that a bunch of similar functions (like allocate and free) which operate similarly for a bunch of different objects are collected into a single page, and expanded with a kind documentation macro. That means that grep’ing the documentation often does not find the object in question.

What I’ve been looking for is a way to find things based upon the objects that are manipulated rather than the actions. This is the first therefore of a series of “what I learnt” blog posts. I expect to come back to this post and add to it. This is also my first post to include disqus links.

X509 Certificate Store

This post is about the X509_STORE_CTX and the X509_STORE. The man pages that are interesting are in the source code at doc/man3 in the files:

Whenever possible, I will point to the master version of the document rather than the specific version, as the version links have historically been sometimes fragile due to web site issues, and anyway, I’d like the master version to get better SEO rank.

The purpose of the X509_STORE is to contain a set of certificates. It provides storage. Some might think it has something to do with buying certificates (i.e. a store that sells certificate), but that’s not it. A great number of web searches no doubt are looking for that, and if you got here because you wanted to buy a certificate, you are in the wrong place.

The major use for the X509_STORE is the X509_verify_cert() function. This function is used in a variety of places to determine if a given certificate is trusted. Certificates arrive by a variety of sources, including:

  • in certificate verification callbacks for TLS (SSL) connections (client certificates and server certificates)

  • it is used in the PKCS7 and CMS verify() routines

The verify process is explained in the section VERIFY OPERATION in verify. The text has been copied to the end of this blog post, it is not terribly useful unfortunately.

(The link from the the man3 pages is broken). It’s generally a good idea to format the HTML version of the man pages and install them some place convenient. The explanation is not very complete, unfortunately.

The X509_STORE_CTX has a cycle of X509_STORE_CTX_new(), .._init(), a series of set/get operations. Then it used by X509_verify_cert() (or a function that calls this). The X509_STORE_CTX can then be recycled by calling .._cleanup() on it, then .._init() again. Or freed with .._free().

In the process of doing it’s work the X509_STORE_CTX may use a series of X509_STORE objects. The files include/openssl/x509_vrf.h, and crypto/x509/x509_vrf.c provide headers and implementation for this. The header file has some short text that explains a very small amount.

The X509_STORE is used to contain certificates that might be needed. The X509_STORE_CTX uses it to create a validated chain. (?)

To do a chain verification, one first creates an X509_STORE_CTX and initializes it with _init(). In general one has one or more certificates that are not known to be trusted, and at least one anchor certificate (usually a self-signed root CA, but not necessarily) which has signed one or more of the given certificates. It is entirely possible for an attacker or malicious peer to provide extra certificates which do not validate, and which are extraneous.

The _init() routine takes an X509_STORE as the trusted anchors, and the untrusted list as the chain. The store is optional, and if provided, it overrides many of the various callbacks and checks listed below with functions from the store.

The chain of certificates to be checked is loaded into ctx->chain, using X509_STORE_CTX_set0_unstrusted(). This takes a STACK_OF(X509), with the requirement that the end certificate be the last item. This requirement comes from specifications.

The set of trust anchors, if not provided as a X509_STORE, can be loaded using X509_STORE_CTX_set0_trusted_stack().

The process is inside X509_verify_cert():

  • build_chain(): this starts with a single, the leaf end-entity certificate, found in the ctx->untrusted, and attempts to extend the chain by looking through the list of attached certs in the ctx->chain.
  • check_chain_extensions(): validates that the found chain matches policy.
  • check_auth_level(): verify that the public keys meet the security level.
  • check_id(): check that the resulting ctx->cert matches the desired host, email or IP address extension.

Some additional checks are then done: CRL checks, check_name_constraints(), RFC3779 path validation.

The author is interested in the CMS functions, specifically:

int CMS_verify(CMS_ContentInfo *cms, STACK_OF(X509) *certs, X509_STORE *store,
            BIO *indata, BIO *out, unsigned int flags);

So with the above understanding, it becomes clearer how to use this function. The store is the list anchors, and the certs list is the list of untrusted artifacts.

In cms_smime.c, the cms_signerinfo_verify_cert() function goes through the signers in the CMS structure. This uses the X509_STORE_CTX mechanism. First it is allocated, then the store is initialized with the provided store of anchors, a stack of certificates from the CMS structure, and the certificate that used to sign.

VERIFY OPERATION

Firstly a certificate chain is built up starting from the supplied certificate and ending in the root CA. It is an error if the whole chain cannot be built up. The chain is built up by looking up the issuers certificate of the current certificate. If a certificate is found which is its own issuer it is assumed to be the root CA.

The process of ‘looking up the issuers certificate’ itself involves a number of steps. After all certificates whose subject name matches the issuer name of the current certificate are subject to further tests. The relevant authority key identifier components of the current certificate (if present) must match the subject key identifier (if present) and issuer and serial number of the candidate issuer, in addition the keyUsage extension of the candidate issuer (if present) must permit certificate signing.

The lookup first looks in the list of untrusted certificates and if no match is found the remaining lookups are from the trusted certificates. The root CA is always looked up in the trusted certificate list: if the certificate to verify is a root certificate then an exact match must be found in the trusted list.

The second operation is to check every untrusted certificate’s extensions for consistency with the supplied purpose. If the -purpose option is not included then no checks are done. The supplied or “leaf” certificate must have extensions compatible with the supplied purpose and all other certificates must also be valid CA certificates. The precise extensions required are described in more detail in the CERTIFICATE EXTENSIONS section of the x509 utility.

The third operation is to check the trust settings on the root CA. The root CA should be trusted for the supplied purpose. For compatibility with previous versions of OpenSSL, a certificate with no trust settings is considered to be valid for all purposes.

The final operation is to check the validity of the certificate chain. The validity period is checked against the current system time and the notBefore and notAfter dates in the certificate. The certificate signatures are also checked at this point.

If all operations complete successfully then certificate is considered valid. If any operation fails then the certificate is not valid.

REFERENCES

This page has covered the following functions:

X509_STORE_CTX *X509_STORE_CTX_new(void);
void X509_STORE_CTX_cleanup(X509_STORE_CTX *ctx);
void X509_STORE_CTX_free(X509_STORE_CTX *ctx);
int X509_STORE_CTX_init(X509_STORE_CTX *ctx, X509_STORE *store,
                        X509 *x509, STACK_OF(X509) *chain);
void X509_STORE_CTX_set0_trusted_stack(X509_STORE_CTX *ctx, STACK_OF(X509) *sk);
void X509_STORE_CTX_set0_untrusted(X509_STORE_CTX *ctx, STACK_OF(X509) *sk);

As a note to self for future edits, this page has not covered:

void X509_STORE_CTX_set_cert(X509_STORE_CTX *ctx, X509 *x);
STACK_OF(X509) *X509_STORE_CTX_get0_chain(X609_STORE_CTX *ctx);
void X509_STORE_CTX_set0_verified_chain(X509_STORE_CTX *ctx, STACK_OF(X509) *chain);
void X509_STORE_CTX_set0_crls(X509_STORE_CTX *ctx, STACK_OF(X509_CRL) *sk);

X509_VERIFY_PARAM *X509_STORE_CTX_get0_param(X509_STORE_CTX *ctx);
void X509_STORE_CTX_set0_param(X509_STORE_CTX *ctx, X509_VERIFY_PARAM *param);
int X509_STORE_CTX_set_default(X509_STORE_CTX *ctx, const char *name);

STACK_OF(X509)* X509_STORE_CTX_get0_untrusted(X509_STORE_CTX *ctx);

int X509_STORE_CTX_get_num_untrusted(X509_STORE_CTX *ctx);

typedef int (*X509_STORE_CTX_verify_fn)(X509_STORE_CTX *);
void X509_STORE_CTX_set_verify(X509_STORE_CTX *ctx, X509_STORE_CTX_verify_fn verify);


X509_STORE *X509_STORE_new(void);
void X509_STORE_free(X509_STORE *v);
int X509_STORE_lock(X509_STORE *v);
int X509_STORE_unlock(X509_STORE *v);
int X509_STORE_up_ref(X509_STORE *v);
Written on February 18, 2019