Moved to https://github.com/containerd/containerd/tree/master/pkg/cri . If you wish to submit issues/PRs, please submit to https://github.com/containerd/containerd
Home Page: https://github.com/containerd/containerd/tree/master/pkg/cri
License: Apache License 2.0
crictl is a CLI to interact with CRI directly for debugging purpose.
We should add a document to tell user/developer how to use crictl to help them debug.
We need a Makefile to manage the build, test, package and release process.
Currently, the network teardown logic assumes that once the network namespace is gone, we don't need to teardown the network again.
However, it doesn't work well with current implementation of cri-containerd. Currently, cri-containerd creates a sandbox container for each sandbox and use the network namespace of this container. But the problem is that, once the container dies, we'll not be able to find the network namespace by /proc/${PID}/ns/net.
So based on current logic, once the sandbox container dies itself, we'll not be able to find the network namespace, thus not teardown network for it. This will cause resource leakage.
We need to maintain a permanent network namespace for each sandbox, which is similar with cri-o.
We should remove the permanent network namespace as soon as we successfully tear down the network, so as to avoid multiple teardown to the same network in most cases.
/cc @xlgao-zju
Privileged is initially a Docker concept. Docker grants a group of permissions and mounts a group of directories into a privileged pod.
In Kubernetes, we should define our own Privileged kubernetes/kubernetes#44503. For now, we could do whatever docker is doing.
Things to do:
Privileged container./cc @heartlock
The initial Makefile is merged in #7.
Currently, we already have lint and fmt verification. We need a boilerplate verification.
We should enable travis to run presubmit build and unit test.
Containerd is still making changes to API, e.g. containerd/containerd#1047, containerd/containerd#1062.
We should upgrade once more after they finalize the api.
A new CRI function UpdateContainerResources is added to update container resource constraints kubernetes/kubernetes#46105.
This is relatively lower priority for now, because Kubelet hasn't started using it yet.
However, since containerd already has corresponding interface, it's easy to do. So let's support it. :)
@kubernetes-incubator/maintainers-cri-containerd
Currently, when stopping a container, currently we send SIGTERM.
In the new image spec, a StopSignal field is added. After updating containerd, we should send the stop signal in the image config if it is specified.
In InitCNI, the first parameter "pluginDir " means the path of cni config file.
"cniDirs" means the path of bin.
func NewCRIContainerdService(containerdEndpoint, rootDir, networkPluginBinDir, networkPluginConfDir,
streamAddress, streamPort string) (CRIContainerdService, error) {
......
netPlugin, err := ocicni.InitCNI(networkPluginBinDir, networkPluginConfDir) <--here
func InitCNI(pluginDir string, cniDirs ...string) (CNIPlugin, error) {
plugin := probeNetworkPluginsWithVendorCNIDirPrefix(pluginDir, cniDirs, "")
Currently we are using k8s.io.
However, containerd will enforce dns label. containerd/containerd#1059
We should get rid of the dot on our side.
https://github.com/kubernetes/repo-infra is great tool.
Now we copied the boilerplate verification part into the repo, but removed the other go code verification.
It would be better to consolidate this in the future and only use repo-infra when it is ready for our use case.
Several related issue:
According to CRI, container runtime is responsible to provide container metrics, including:
See here
Containerd should have provided all those information:
We should support these container metrics.
/cc @kubernetes-incubator/maintainers-cri-containerd
cri-containerd already supports basic container/sandbox lifecycle and image management today. However, there are still many missing pieces. List them here with relative proprieties:
ExecSync is very useful for testing, so prioritize it./etc/resolv.conf. (Issue: #28, PR: #50)/etc/hosts. (PR: #60)/dev/shm. (PR: #67)containerd and cri-containerd are in runtime cgroup. (Issue: #181, PR: containerd/containerd#1443, #184)cri-containerd and containerd restart.We should cleanup logs and errors:
%+v for struct, %q for string, and %v for error;/cc @mikebrow
After #113, cri-containerd will be using containerd client.
However, there are still some image specific code needs to cleanup, e.g. getImageInfo, localResolve etc.
We should add required functions into containerd client, and clean those functions up.
/cc @kubernetes-incubator/maintainers-cri-containerd
@abhinandanpb
To better unit test functions of cri-containerd, we need a group of fake containerd services (similar with fake docker client)
Containerd now has 4 kinds of services:
We can add a fake execution service first, because the execution api is more stable. And add others later when image management code is added and needs test.
/cc @kubernetes-incubator/maintainers-cri-containerd
@heartlock Are you interested in this? This is a good start to get familiar with containerd and the code base.
Currently, we pass the grpc handler context to containerd client directly.
Once the context is cancelled by the client, no functions calls could be made to containerd client.
The good thing is that, we can avoid proceeding when user want to cancel the context. However, the bad thing is that we the cleanup functions in defer will not be able to run, either.
Note the problem here, so that we could revisit this in the future.
Add cri-containerd version.
We should version cri-containerd based on the repo tag and commit number. The version will be used:
--version output.Containerd is adding a new client, which will make integration and our unit test much easier.
We should use it after it lands.
We need to design, implement and document a practical way to create a Kubernetes cluster using containerd as container runtime.
kubeadm seems to be the way to go, and https://www.projectatomic.io/blog/2017/06/using-kubeadm-with-cri-o/ is very good reference.
We can start to run some cluster e2e test after this is done.
/cc @kubernetes-incubator/maintainers-cri-containerd
We should put containerd-shim into pod cgroup, so that we could charge the resource usage to pod instead.
Ref containerd/containerd#1032 and containerd/containerd#1134.
/cc @lanchongyizu
Kubernetes 1.7 code freeze started, we should update CRI version to the newest version kubernetes/kubernetes#45614.
Currently, containerd only supports schema 2 manifest.
For cri-containerd, not supporting schema 1 means:
Regarding to 1), it is fine for our alpha release (this quarter), but it should be fixed before beta (probably the end of the year).
For 2), we need to rebuild our schema 1 system container image.
generateSandboxContainerSpec() is currently filtering all annotations received through the CRI api vs copying them to the runtime spec. Consider if some of the annotations should be filtered or if all should be copied over. For example:
annotations := config.GetAnnotations()
for key, value := range annotations {
g.AddAnnotation(key, value)
}
Similarly, labels received over CRI are not being processed / handed over to containerd.
Consider the patterns enabled/being used for kubernetes when connected to dockershim and CRI-O, should we be unique here?
CRI expects the runtime to handle container networking.
As is expected, containerd itself doesn't provide container networking. We should integrate cri-containerd with kubernetes network plugin to initialize the network namespace.
Things need to do:
namespaceGetter and portMappingGetter. (See https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockershim/docker_service.go#L119-L137)/cc @xlgao-zju
Ref
We should change vendored Sirupsen to sirupsen after dependencies are resolved.
Currently we are using nsenter for both CNI plugin and port forword.
Ideally, we should be able to use NetNS.Do to run code inside a specific network namespace.
However, golang has an issue that it may spawn new threads from locked thread, which will inherit the network namespace. golang/go#20676
We should be able to get rid of all nsenters after this is fixed.
Containerd should have already supported pull image authentication. containerd/containerd#783
We should figure it out whether it's enough for us and use it.
Error happened when run make lint
root@ubuntu:/home/vcap/go-project/src/github.com/kubernetes-incubator/cri-containerd# make lint
checking lint
for directory ./cmd ...
for directory ./cmd/cri-containerd ...
WARNING: deadline exceeded by linter interfacer on ./cmd/cri-containerd (try increasing --deadline)
WARNING: deadline exceeded by linter gosimple on ./cmd/cri-containerd (try increasing --deadline)
WARNING: deadline exceeded by linter structcheck on ./cmd/cri-containerd (try increasing --deadline)
Makefile:31: recipe for target 'lint' failed
make: *** [lint] Error 2
check so slow that exceed deadline
@mikebrow @Random-Liu
Update ocicni vendoring to master once cri-o/ocicni#1 is merged.
Many of our unit tests rely on fake containerd services. Unit test does give us some values:
However, since containerd is still in alpha, they are still getting feedbacks and may change api again and again, it's a pain and a bit waste of effort to maintain the fake services up to date.
We decided to temporarily remove the unit tests which use fake services, and rely on the integration test (CRI validation test and node e2e test soon) for now.
We'll add the unit test back after:
The commit before removing unit test is cbd936b.
It's too early to add a developer guide, but we should add one in the future when the build and development process is finalized.
There are several restart recovery problems with current cri-containerd:
To fix this, we should recover/reconcile state during cri-containerd start or after containerd restart and reconnect.
There are 3 kinds of internal state:
/cc @kubernetes-incubator/maintainers-cri-containerd
Kubernetes 1.8 and runc both support it, we should update CRI version and support it.
Currently, we stop sandbox container and remove permanent network namespace when stop sandbox.
When remove sandbox, we check whether sandbox container is still running, but we should also check whether network namespace is properly removed.
Previously, Docker maintains a resolv.conf for containers sharing the same network namespace.
Now, when using containerd, we need to do these ourselves. We need to:
HostNetwork. And initialize resolv.conf with DNSOptions in PodSandboxConfig.This logic is similar with cri-o. (See https://github.com/kubernetes-incubator/cri-o/blob/master/server/sandbox_run.go#L144-L159)
/cc @xlgao-zju
We added make test-e2e-node in #145.
We should document that in the test document https://github.com/kubernetes-incubator/cri-containerd/blob/master/docs/testing.md.
We need a fake OS for unit test, similar with https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/container/testing/os.go.
Kubernetes enabled Pid namespace sharing inside a pod with Docker 1.13+.
However, some customers reported that they could not run their container, because their containers made assumptions about the pid 1.
Before resolving this issue, we could not use pid namespace sharing in production. In cri-containerd, we should also disable it by default, and probably introduce a flag to enable it.
We need an example cni config in user guide.
In current implementation, we let container inside a sandbox share the same uts namespace. This is what we want in Kubernetes kubernetes/kubernetes#1615.
However, docker doesn't support UTS namespace sharing. We should figure out:
The previous container concept in containerd is changed to task.
In the refactoring PR, I've changed most of them to task.
However, there are some left, we should clean them up.
Required Internal Services:
runtime cgroup. (See dockershim container manager)Container Runtime Interface Functions:
This is covered by containerd now. See here.
The execution api of containerd is updated just now containerd/containerd#894.
This is a good move for us, we could potentially checkpoint state-independent data into the container labels, like what we are doing for docker today.
However, we still need to do checkpoint for the state-dependent data, such as exit status, container state, timestamp.
We should update containerd version and use the new execution api. This basically involves:
The image management code is merged #21.
However, we don't have unit test for it, because:
We should:
ListImages and ImageStatus purely based on metadata store, it's testable.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.