I am documenting here the process of how to use a private Azure Container Registry with your Kubernetes environment. Azure Container Registry can also be used as a private repository location for your Docker images you do not wish to publish. The registry can be replicated across regions automatically, keeping the images close to the compute. Setting up a private Docker Registry on Flashblade S3 allows team members to share environments and better collaborate on various AI projects.Īs the storage for an AI Data Hub, FlashBlade can provide scalable, performant storage for not only the hot tier of datasets that data scientists are training on, but also for ancillary components like a Docker registry.Using Azure Container Registry in KubernetesĪzure Container Registry is a private Docker registry for your Docker images. CloudBerry) to get a file-system like view of the registry bucket. Modification 4: If your team is highly visual, you can use an object storage explorer tools like MSP360 (fka. Alternatively, it’s possible to launch the registry as a Kubernetes pod and use an ingress service to forward traffic to the registry. Modification 3: In this example, we configured a Docker registry outside Kubernetes so that the registry can be shared across multiple clusters. You can find example configuration steps in Pure’s Registry-as-a-Service white paper. Modification 2: As previously mentioned, it’s possible to make your registry more secure by including a self-signed certificate. Users can now push images to the registry at :5000. The /etc/hosts file on the node hosting the registry server can be used to map to map the node’s IP address to a new domain name. Here are some of our favorite registry enhancements. In the Rancher GUI for a cluster: Workloads -> Deploy -> select an image from the registry for the Docker Image field Optional Modifications Considering that the Secret Access Key will be needed to configure the registry, it is recommended that it be downloaded and saved as a JSON or CSV file. Note: when you create the Access Key, you will be provided with a Secret Access Key, which is only accessible at the time of Access Key creation. You can then add a “registry” bucket and a new user for this Account. In the FlashBlade GUI, navigate to Storage > Object Store and create a new Account. Here, we’ll demonstrate how to do the setup in the FlashBlade GUI, but our colleague Joshua R obinson wrote a python script that automates the creation of S3 users, keys, and buckets. Configure FlashBladeįirst, we’ll discuss how to set up an S3 bucket on FlashBlade object store to be the backend for the registry. We’ll also provide example usage of the registry. An insecure registry is a quick way to configure a registry in a lab environment that’s on a secure private network.Īt a high level, the configuration steps include: setting up an S3 bucket on FlashBlade, configuring the node that hosts the registry server, and launching the server. Obtain a TLS certificate from a 3rd-party certificate authority – official recommendation from DockerĮach of these options require some additional configuration.Use HTTP ( “insecure-registry” mode ) – method followed bellow.There are three options for securing a registry: Note: In this post, we set up an “insecure” Private Docker Registry since our cluster is on a tightly-controlled network. In this blog post, we describe how to configure a private Docker registry on FlashBlade S3. It also lowers overall costs by avoiding silos of unused capacity and by sharing hardware across performance-sensitive and colder jobs. Storing more components of your AI platform on centralized storage, such as FlashBlade S3, results in simpler management and operations. Many components of an AI data hub require storage. It can also help simplify your Artificial Intelligence (AI) deployment if you store your Docker registry on the same machine where your data scientist’s training datasets are saved. You may want to set up a private Docker registry for environments that do not have internet connection or do not use DockerHub. Doing so simplifies sharing between team members as well as across compute nodes, like GPU servers. However, the total number of Docker images owned by a team can balloon quickly.Īs teams work toward repeatable, scalable tests, it becomes increasingly important to manage Docker images centrally. A common solution is to constantly create development environments by saving new Docker images.
This is the second post in a multi-part series by Bikash Roy Choudhury and Emily Watkins, where we discuss how to configure a Kubernetes-based AI Data Hub for data scientists.įor data science experiments, there’s always a new library–or a new version of an existing library–to test, and data scientists are most likely juggling six other libraries they want or need specific versions of.