icfpcontest2020 / dockerfiles Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
I've made a simple starter kit here:
https://github.com/prozacchiwawa/icfp2020-fsharp-starter-pack
It uses the existing csharp platform, so there's no Dockerfile needed.
It's necessary to run setup-pkgbin.sh to get a local copy of FSharp.Core. (error output is ok actually).
I've only very slightly tested this but seems ok.
PHP was declared as the programming tool of choice for discriminating hackers in 2017.
Hi,
I've seen that for example because of #8 some non standard-libraries were added to the haskell image. Can we expect that you will always extend the corresponding images if users request additional libraries? Or will it depend on the library and the reasoning?
What I've done:
submission
The repository is [email protected]:Vlad-Shcherbina/icfpc2020-tbd.git
.
The submission checklist is in the following state:
✅ Create a private Git repository
✅ Register your team
---- YOU ARE HERE ---
☑️ Copy the deploy key and add it to your repository (?)
☑️ Create a branch called submission
☑️ Download a starter kit for your language of choice
☑️ If there's none, file an issue
☑️ Commit and push the starter kit to submission
☑️ Wait for your submission to appear below (<= 10 min)
No submission appears in the table.
GitHub shows that this deploy key was recently used, so apparently your bot is scraping the repository.
Please consider adding this language or platform.
I might possibly send a PR with a starter kit (and can try to make Dockerfile - but have no experience).
Please consider adding this language or platform.
As per this tweet
I might possibly send a PR with a Dockerfile and a starter kit.... or just dump the files into this issue, as I did for R, because I can't get my head around how to submit a pull request that works properly.
As initially proposed in #11, some platforms now support customizing the build/run process with participant-supplied shell scripts (e.g. #21). But other platforms don't.
To avoid confusing participants, I propose to align the build/run protocol among all platforms. In other words, all Dockerfile
in this repository should be identical except for the base image reference (e.g. icfpcontest2020/python
).
I imagine that such Dockerfile
would be like this:
FROM icfpcontest2020/${platform}-build AS build # This line varies by platforms
RUN mkdir -p /app
WORKDIR /build
COPY . .
RUN ./build.sh # This script should build the solution and install it to /app
FROM icfpcontest2020/${platform}-run # This line varies by platforms
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["./run.sh"]
Also, if we're fine with reusing build-time images as run-time images, Dockerfile
can be simplied to something like the following. Actually this looks better to me, unless it has big implication to the evaluation infrastructure, because the difference of build-time dependencies and run-time dependencies is often tricky and hard to debug.
FROM icfpcontest2020/${platform} # This is the only line that varies by platforms
RUN mkdir -p /app
WORKDIR /build
COPY . .
RUN ./build.sh # This script should build the solution
WORKDIR /app
ENTRYPOINT ["./run.sh"]
As you can see, pull requests often come in pairs: one to dockerfiles and one to whatever-starter-kit. It should be one atomic change. Furthermore, it's inconvenient for contributors to propose new platforms without you creating a new empty starter kit repo first.
Consider merging all this stuff into a monorepo.
It will make getting started a bit harder (one can't just clone favorite-language-starter-kit, has to copy a sub-directory), but I think it's worth it.
Please consider adding this language or platform.
I looked Dockerfile
s in this repository and find that
only single source is copied into container image.
Current submission system favors specific languages that have ability to pack multiple sources into single source (for example, JavaScript)
Is the single source constraint hard?
If so, is there any restriction on the size of the main source?
What I've done:
submission
Here's the problem:
I got the initial submission
branch working but wanted to test the submissions/*
style so my team mates and I can submit various directions without stepping on each other's toes. I pushed submissions/starter
as an exact copy of the submission
branch that previously worked (as well as trying a number of other non-identical branches) and none of them (except the original) have shown up on the submissions page. This is for team OMG ICFP FTW.
What might I be doing wrong?
What I've done:
submission
Here's the problem:
[WHAT WENT WRONG]
Log from failed build ^^
should the start kit stuff (app/, run.sh, build.sh) be in the root of the repo or under the starter_kit_* dir in my repo?
Edit: I've just seen the .platform
file as well, committed that, waiting on the output
Are repositories cloned recursively such that we can, for example, include header-only libraries in submodules?
My R solution got an implicit mention in last year's summary (as a team where I used the neural network of my partner and kids to help solve puzzles). It would be great to be able to use R again this time around.
Being able to use Shiny as well (for quickly designing UI that work as Web applications) would be extremely helpful:
Ruby was declared as a fine programming tool for many applications in 2018.
What I've done:
submission
Here's the problem:
Cannot find revision of the default branch 'refs/heads/submission' of vcs root 'jetbrains.git'
first time through I lost the .dot files from the starter kit, but then I read about needing .platform and I committed/pushed .platform to my submission branch. Still same issue on submission log.
I did add the deploy key.
[WHAT WENT WRONG]
my repo: https://github.com/GeePokey/Canivsar2020
Currently platform contributors verify Docker builds locally. This is probably okay before the contest, but I'm afraid that it's undesirable during the contest because any change to Dockerfiles may break other team's solutions.
It is great if we could set up CI to verify Dockerfiles on pull requests. It would do something like:
I'd like to have Go support.
I'm sending a PR soon.
Thanks,
Vlad.
What I've done:
submission
Here's the problem:
I got the following error: 405 Client Error: Method Not Allowed.
It's the first time I got the error. So far I (almost) only changed stuff in my developer-folder. So the failed-test was basically the python starter-kit. I guess there is a problem with the server? Here is the corresponding test log:
ServerUrl: http://server:12345; PlayerKey: 4545917190182324706
<Response [405]>
Traceback (most recent call last):
File "app/main.py", line 14, in <module>
main()
File "app/main.py", line 8, in main
utils.start(server_url=sys.argv[1],
File "/solution/app/icfp_lib/utils.py", line 6, in start
res.raise_for_status()
File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 405 Client Error: Method Not Allowed for url: http://server:12345/?playerKey=4545917190182324706
I wasn't sure if I should create an issue for this. Are you providing information about the server status somewhere? So that we do not create unnecessary issues if you already know about problems / planned reconfiguring of the server etc?
Please consider adding this language or platform.
Bazel is a language-agnostic build system. My team has been using the system to build C++ programs in recent ICFP contests (2017, 2018, 2019). We'd like to use it in this year's contest as well.
However Bazel downloads some language toolchains during build for reproducibility. For example, on buliding Java programs, Bazel downloads OpenJDK of a fixed version. However README says there is no Internet access during build.
Could you give me guidance on how to proceed?
Please consider adding this language or platform.
What can I do to support a PR with a Dockerfile and a starter kit?
Repository URL: [email protected]:Vlad-Shcherbina/icfpc2020-tbd.git
The day before yesterday I submitted, among others, the commit bd58c6
. Test server successfully ran it.
However, yesterday it ran it 7 more times without any action on my side.
Just letting you know.
Would you mind clarifying the license of this repository? I'd like to contribute and will be comfortable if the license is clear.
Please consider adding this language or platform.
It should be a small modification to starterkit-javascript to have build.sh run the TypeScript compiler.
What I've done:
submission
Here's the problem:
I'm using python. The starter-kit works. But when I replace the post command with:
import requests
res = requests.post("https://icfpc2020-api.testkontur.ru/aliens/send?apiKey=<my-api-key>", data="string")
then the testing stages fails with:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 308, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7feabcc7e490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='icfpc2020-api.testkontur.ru', port=443): Max retries exceeded with url: /aliens/send?apiKey=<my-api-key> (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7feabcc7e490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "app/main.py", line 14, in <module>
main()
File "app/main.py", line 8, in main
utils.start(server_url=sys.argv[1],
File "/solution/app/icfp_lib/utils.py", line 10, in start
res = requests.post(url, data=data)
File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='icfpc2020-api.testkontur.ru', port=443): Max retries exceeded with url: /aliens/send?apiKey=<my-api-key>(Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7feabcc7e490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
Am I doing something wrong?
It is mentioned in other issues that the organizers are ok with pre-built binaries in the solutions.
This, however, does not help the jvm languages as the bash image does not include java.
Providing several images containing typical jvm build systems (maven/gradle/sbt/etc.) also makes little sense, as they all are built around having an internet connection which is disallowed during build. Furthermore, providing dependencies for e.g. maven without internet connection is pretty hard, non-trivial and ugly.
I hereby propose to include a jvm-generic image that accepts any app.jar
as the program source and runs it (essentially the java image minus the actual javac invocation). This is essentially the same as "submitting pre-built binary" but for jvm.
Please consider adding this language or platform.
Racket https://racket-lang.org
I might possibly send a PR with a Dockerfile and a starter kit.
https://hub.docker.com/r/racket/racket/
based on repo https://github.com/jackfirth/racket-docker
If I understand README correctly, this repository is going to host Dockerfiles, each of which corresponds to a single programming language.
But what if a solution consists of multiple binaries of different programming languages? For example, what if it a solution consists of C++ binary and Rust binary interacting each other with IPC?
OCaml was declared as very suitable for rapid prototyping in 2018.
This probably also applies to other languages in varying extents, but I'd like to draw attention to the fact that a lot of commonly used functionality in Haskell is implemented outside of the "standard library" and thus currently missing from the corresponding Docker image. Unless the submissions are supposed to use only the standard prelude, I'd like to ask whether some commonly used Haskell libraries could be added to the image. If you agree to include include a certain set of libraries, I could make a PR with an extended Dockerfile.
As a concrete proposal, here are some packages the team I'm part of (udfew) used frequently in the last years:
My team is a bit concerned about how dependencies for Python are handled. I understand the advertised process is to make a PR for the python
platform to add to the requirements.txt
. But we don't know which libraries are needed before hand, we actually often learn new libraries during the contest. How is this going to scale for a large number of teams and vast number of possible python libraries? How could version conflicts be handled?
I understand an ad-hoc installation of custom requirements.txt
, or even better with a proper setuptools setup.py
wont work due lack of internet connectivity.
What's the way out here? I could envision having ready-made site-packages
folders in the submission branch, e.g. taken from a virtualenv, but I'm not sure about the limitations of portability.
What I've done:
submission
Here's the problem:
Submission builds fine, but tests failed. Test logs are empty, so there is no way to diagnose error
What I've done:
submission
Here's the problem:
The submissions page shows "Something went wrong. Server answer is 500" as error message. I've seen it work about two hours ago with two submissions visible.
Kotlin is the 4th loved and 6th wanted programming language, according to StackOverflow survey.
What I've done:
We alread have successfull submission with Test failed status.
Made little change to it (sending to server "0111101011011000" and "0111111111111111"), and
Here's the problem:
28 minutes has gone and still no results. Only "Testing" status.
What I've done:
submission
Here's the problem:
We noticed that the submissions page now mentions the branch name, and my teammate @pink-snow got curious how the "submissions/submission" branch would be displayed. So here's what I did:
In a few minutes, a new entry appeared on the site, but it mentions the same SHA-1 and commit message as the commit on the "submission" branch. When I pushed more commits to "submissions/submission", it created more builds of the "submission" branch.
Please don't ban us for this ._.
A library will be needed to talk to an HTTP server in C++. There are likely many options. One I have used is libcurl. The development package (libcurl4-openssl-dev) is already available in the gcc:10.1.0 image, assuming the build process is enhanced to permit linking against a library (#20). But the runtime package (libcurl4) is not available in the debian:buster-20200607-slim image, so the resulting binary will not run.
More generally, it will probably be necessary to create a customized base image as was done with haskell (#8) so packages needed/wanted by various teams can be installed. Packages may need to be added to both the gcc:10.1.0 and debian:buster-20200607-slim images, so I supposed two customized images should be created.
Hard-coding the C++ compiler command line is problematic for a number of reasons:
As an alternative, I recall a past contest where teams could supply two scripts, e.g. "build" and "run". The build script can invoke the compiler with any necessary arguments, or run a tool like "make", etc. The run script can run the program with any necessary arguments, environment variables, etc. If no special arguments are needed, the run script can just be the binary itself. I would also suggest copying an entire directory rather than just a single binary. The directory might contain the run script, the compiled binary, data files, shared libraries, etc.
I'll create a pull request along these lines if you are open to it.
Rust was declared as the programming tool of choice for discriminating hackers in 2018.
Hi,
yesterday I've registered for the contest successfully and already did some stuff to get familiar with the submission system. But I cannot log in again. I'm getting asked for an API key which I do not have. I checked my Emails but I did not receive any Email from the ICFP contest organizers.
To whom can I reach out in this regard?
The help on Deploy keys for GitLab should be this:
Settings → Repository → Deploy Keys [Expand]
It is currently this:
Settings → CI / CD → Deploy Keys
Like in issue #51 I failed to note the API Key.
Team name: The Higher Order of Zeuxis
Thx for the support!
What I've done:
submission
Here's the problem:
[WHAT WENT WRONG]
The build failed with:
[12:28:35] : Step 2/7: replace dockerfile with our version (Command Line)
[12:28:35] : [Step 2/7] Starting: /opt/buildagent/temp/agentTmp/custom_script9028038695576878708
[12:28:35] : [Step 2/7] in directory: /opt/buildagent/work/5c8458c2f44b95ac
[12:28:36] : [Step 2/7] Process exited with code 0
[12:28:36]E: Step 3/7: build docker image (Docker) (24s)
[12:28:36] : [Step 3/7] Starting: docker build -t icfpc-registry.kontur.host/icfpc_submissions/2625c98f56c348e58631d07067ae832d:9a5c6195ee6dbcc935776a63382755a854c2d871 -t 241256198909.dkr.ecr.us-east-2.amazonaws.com/2625c98f56c348e58631d07067ae832d:9a5c6195ee6dbcc935776a63382755a854c2d871 --pull --network none -f Dockerfile .
[12:28:36] : [Step 3/7] in directory: /opt/buildagent/work/5c8458c2f44b95ac
[12:28:36] : [Step 3/7] Sending build context to Docker daemon 16.78MB
[12:28:36] : [Step 3/7]
[12:28:36] : [Step 3/7] Step 1/7 : FROM icfpcontest2020/scala
[12:28:38] : [Step 3/7] latest: Pulling from icfpcontest2020/scala
[12:28:38] : [Step 3/7] 4167d3e14976: Pulling fs layer
[12:28:38] : [Step 3/7] 345cf19c142c: Pulling fs layer
[12:28:38] : [Step 3/7] baeb5b3b9357: Pulling fs layer
[12:28:38] : [Step 3/7] 3b0c85193bfe: Pulling fs layer
[12:28:38] : [Step 3/7] 3b0c85193bfe: Waiting
[12:28:39] : [Step 3/7] 4167d3e14976: Verifying Checksum
[12:28:39] : [Step 3/7] 4167d3e14976: Download complete
[12:28:39] : [Step 3/7] baeb5b3b9357: Download complete
[12:28:39] : [Step 3/7] 4167d3e14976: Pull complete
[12:28:41] : [Step 3/7] 3b0c85193bfe: Verifying Checksum
[12:28:41] : [Step 3/7] 3b0c85193bfe: Download complete
[12:28:45] : [Step 3/7] 345cf19c142c: Verifying Checksum
[12:28:45] : [Step 3/7] 345cf19c142c: Download complete
[12:28:54] : [Step 3/7] 345cf19c142c: Pull complete
[12:28:55] : [Step 3/7] baeb5b3b9357: Pull complete
[12:28:55] : [Step 3/7] 3b0c85193bfe: Pull complete
[12:28:55] : [Step 3/7] Digest: sha256:b12809acaf7fef9f2f424b8e77c03d7ff49b698207f66853fa1bc9cd79e05ceb
[12:28:55] : [Step 3/7] Status: Downloaded newer image for icfpcontest2020/scala:latest
[12:28:55] : [Step 3/7] ---> ce55cb81abab
[12:28:55] : [Step 3/7] Step 2/7 : WORKDIR /solution
[12:28:56] : [Step 3/7] ---> Running in 613d711de15c
[12:28:56] : [Step 3/7] Removing intermediate container 613d711de15c
[12:28:56] : [Step 3/7] ---> 891f5e9be49f
[12:28:56] : [Step 3/7] Step 3/7 : COPY . .
[12:28:57] : [Step 3/7] ---> 9260e3d381aa
[12:28:57] : [Step 3/7] Step 4/7 : RUN chmod +x ./build.sh
[12:28:57] : [Step 3/7] ---> Running in 156b2e05aab0
[12:28:58] : [Step 3/7] Removing intermediate container 156b2e05aab0
[12:28:58] : [Step 3/7] ---> fbdfc711b566
[12:28:58] : [Step 3/7] Step 5/7 : RUN chmod +x ./run.sh
[12:28:58] : [Step 3/7] ---> Running in 665f6be8ee3f
[12:28:59] : [Step 3/7] Removing intermediate container 665f6be8ee3f
[12:28:59] : [Step 3/7] ---> 68bc72545724
[12:28:59] : [Step 3/7] Step 6/7 : RUN ./build.sh
[12:28:59] : [Step 3/7] ---> Running in 8f0cc4b25b14
[12:28:59] : [Step 3/7] #
[12:28:59] : [Step 3/7] # A fatal error has been detected by the Java Runtime Environment:
[12:28:59] : [Step 3/7] #
[12:28:59] : [Step 3/7] # SIGILL (0x4) at pc=0x00007f2d59f098f2, pid=20, tid=21
[12:28:59] : [Step 3/7] #
[12:28:59] : [Step 3/7] # JRE version: (14.0+33) (build )
[12:28:59] : [Step 3/7] # Java VM: OpenJDK 64-Bit Server VM (14-ea+33, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
[12:28:59] : [Step 3/7] # Problematic frame:
[12:28:59] : [Step 3/7] # v ~StubRoutines::call_stub
[12:28:59] : [Step 3/7] #
[12:28:59] : [Step 3/7] # Core dump will be written. Default location: /solution/core.20
[12:28:59] : [Step 3/7] #
[12:28:59] : [Step 3/7] # An error report file with more information is saved as:
[12:28:59] : [Step 3/7] # /solution/hs_err_pid20.log
[12:29:00] : [Step 3/7] #
[12:29:00] : [Step 3/7] #
[12:29:00] : [Step 3/7] [91m/usr/local/share/scala/bin/scalac: line 25: 20 Aborted (core dumped) "$@"
[12:29:00]W: [Step 3/7] The command '/bin/sh -c ./build.sh' returned a non-zero code: 134
[12:29:00] : [Step 3/7] [0m
[12:29:00]W: [Step 3/7] Process exited with code 134
[12:29:00]E: [Step 3/7] Process exited with code 134 (Step: build docker image (Docker))
[12:29:00]E: [Step 3/7] JVM crashed
[12:29:00]E: [Step 3/7] Step build docker image (Docker) failed
A fresh checkout of the same repository and branch works perfectly fine for me:
❯ docker build --pull --network none --file ../dockerfiles/dockerfiles/scala/Dockerfile .
Sending build context to Docker daemon 32.13MB
Step 1/7 : FROM icfpcontest2020/scala
latest: Pulling from icfpcontest2020/scala
Digest: sha256:b12809acaf7fef9f2f424b8e77c03d7ff49b698207f66853fa1bc9cd79e05ceb
Status: Image is up to date for icfpcontest2020/scala:latest
---> ce55cb81abab
Step 2/7 : WORKDIR /solution
---> Using cache
---> 6f982051366f
Step 3/7 : COPY . .
---> Using cache
---> a3b12382adf7
Step 4/7 : RUN chmod +x ./build.sh
---> Using cache
---> 89d7f9e74644
Step 5/7 : RUN chmod +x ./run.sh
---> Using cache
---> 5179e1afac44
Step 6/7 : RUN ./build.sh
---> Using cache
---> 729e4b470a6d
Step 7/7 : ENTRYPOINT ["./run.sh"]
---> Using cache
---> 179867e3968a
Successfully built 179867e3968a
This is very strange. It is starting off with the same image ce55cb81abab but then something is different. Any idea what could be different?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.