Mac armv8: Docker energyplus input_file: File does not exist: energyplus

[rh0dium]

Mac armv8: Docker energyplus input_file: File does not exist: energyplus

[rh0dium]

@ckm47 I can confirm your docker setting changes will make it work. I was able to narrow this a bit more simply shifting from VirtioFS to gPRC FUSE (which disables Rosetta) will make this work. Then I shifted back to VirtioFS but keeping Rosetta disabled. Still works.

tldr; Uncheck Rosetta for x86/amd64 emulation on Apple Silicon under Docker Desktop Settings will FIX this issue. This is not ideal but will work.

FYI - Verified this using Docker Desktop v4.27.2

[jmarrec]

I didn't really want to have to debug that stuff to be honest (when you start having to dig into docker and rosetta troubles you're getting pretty far outside of E+ internals), but I still did.

Turns out the issue is that I rely on CLI11's (circa 2.3.2) interpretation of argc/argv, which for linux reads /proc/self/cmdline separated by null terminators.

https://github.com/CLIUtils/CLI11/blob/985a19f3860be0cba87354336f6588494b20111c/include/CLI/impl/Argv_inl.hpp#L141

When you run under rosetta, you end up with a duplicated program path, looks like rosetta is trying to remove itself from the command line but duplicating it. So wheen you call energyplus --help what cmdline has is energyplus energyplus --help

This is essentially the same issue as described here: docker/for-mac#7058

Someone created a tool to fix it, cf comment: docker/for-mac#7058 (comment) and https://github.com/norio-nomura/re-register-rosetta

This is an implementation that follows what's written in the doc https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta#3978496

After running docker run --platform=linux/arm64 --privileged --rm ghcr.io/norio-nomura/re-register-rosetta I can successfully run docker run -it --rm --platform linux/amd64 -v /Users/julien/Software/OSCLI11:/home/OSCLI11 nrel/energyplus:23.2.0 bash and use energyplus

Mind that the end of the linked issue says it's a bug in Rosetta and it's tracked by rdar://123115850 (that's Apple Radar, their internal bug tracking system, so you have no way to see if that's being addressed or not).

[jmarrec]

Also note that I have a mac M1 Max with the same exact system version and it works perfectly fine on my machine

[jmarrec]

Porting over some debugging @rh0dium did here: NREL/OpenStudio#5057 (comment)

>  /usr/local/openstudio-3.7.0/EnergyPlus/energyplus -w ../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw --debug-cli       

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/usr/local/openstudio-3.7.0/EnergyPlus/Energy+.idd',

runEPMacro = false,
prefixOutName = eplus,

state.dataGlobal->runReadVars=false,
state.dataGlobal->outputEpJSONConversion=false,
state.dataGlobal->outputEpJSONConversionOnly=false,

suffixType=L,

state.dataGlobal->numThread=1,
state.files.inputWeatherFilePath.filePath='../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw',
state.dataStrGlobals->inputFilePath='/usr/local/openstudio-3.7.0/EnergyPlus/energyplus',

The fact that state.dataStrGlobals->inputFilePath='/usr/local/openstudio-3.7.0/EnergyPlus/energyplus' looks VERY strange to me. It's supposed to be just in.idf when the IDF file is omitted.

On my machine:

$ /usr/local/openstudio-3.7.0/EnergyPlus/energyplus -w ../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw --debug-cli

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/usr/local/openstudio-3.7.0/EnergyPlus/Energy+.idd',

runEPMacro = false,
prefixOutName = eplus,

state.dataGlobal->runReadVars=false,
state.dataGlobal->outputEpJSONConversion=false,
state.dataGlobal->outputEpJSONConversionOnly=false,

suffixType=L,

state.dataGlobal->numThread=1,
state.files.inputWeatherFilePath.filePath='../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw',
state.dataStrGlobals->inputFilePath='in.idf',

Somehow is looks like it understands /usr/local/openstudio-3.7.0/EnergyPlus/energyplus as the first argument itself instead of being the program name?!

[jmarrec]

On my M1 Max, testing the official E+ docker image (https://hub.docker.com/r/nrel/energyplus/tags , from https://github.com/NREL/docker-energyplus)

$ docker run -it --rm --platform linux/amd64 nrel/energyplus:23.2.0 bash

root@32d34102cfbf:/# energyplus --debug-cli

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/EnergyPlus-23.2.0-7636e6b3e9-Linux-Ubuntu20.04-x86_64/Energy+.idd',

runEPMacro = false,
prefixOutName = eplus,

state.dataGlobal->runReadVars=false,
state.dataGlobal->outputEpJSONConversion=false,
state.dataGlobal->outputEpJSONConversionOnly=false,

suffixType=L,

state.dataGlobal->numThread=1,
state.files.inputWeatherFilePath.filePath='in.epw',
state.dataStrGlobals->inputFilePath='in.idf',

Then running a simulation works fine.

cd home
apt update && apt install curl
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden-NREL.724666_TMY3.epw
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
cp 1ZoneUncontrolled.idf in.idf
energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf 
[...]
EnergyPlus Completed Successfully.

@rh0dium, can you confirm this also fails for you please?

[rh0dium]

@jmarrec

This fails for me.

❯ docker run -it --rm --platform linux/amd64 nrel/energyplus:23.2.0 bash
root@b3439f0a1bc8:/# energyplus --debug-cli
input_file: File does not exist: energyplus
Run with --help for more information.
root@b3439f0a1bc8:/# 

Same with this

>> cd home
>> apt update && apt install curl
>> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden->> NREL.724666_TMY3.epw
>> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
>> cp 1ZoneUncontrolled.idf in.idf
>> energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf 
input_file: File does not exist: energyplus
Run with --help for more information.

Can you confirm the following?

Docker Desktop 4.25.2 (129061)
Engine: 24.0.6
Compose: v2.23.0-desktop.1

This seems to look similar to #10308 ¯_(ツ)_/¯

[rh0dium]

And even more bizzare is that this work natively..

> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden->> NREL.724666_TMY3.epw
> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
> cp 1ZoneUncontrolled.idf in.idf
>  /Applications/OpenStudio-3.7.0/EnergyPlus/energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf
[...]
EnergyPlus Completed Successfully.

[ckm47]

I was running into similar issues and was able to solve it by changing the following docker settings:

• Use Virtualization framework : false
• File sharing implementation: gRPC FUSE
• Use Rosetta for x86/amd64 emulation on Apple Silicon: false

Configuration:

• M2 macbook Sonoma 14.2.1 (23C71)
• Docker Desktop 4.26.1 (131620)
• OpenStudio-3.7.0+d5269793f1-Ubuntu-20.04-x86_64.deb

[rh0dium]

Couple more tests with Rosetta enabled. I wanted to verify that this was with EnergyPlus and not a Mac docker issue.

cd home
apt update && apt install curl
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden-NREL.724666_TMY3.epw
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
cp 1ZoneUncontrolled.idf in.idf
energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf 

Version 22.1.0: WORKS

Version 22.2.0: WORKS

Version 23.1.0 WORKS

Version 23.2.0 FAILS

So this isn't a docker issue this is something that changed between 23.1.0 and 23.2.0.

Only 1600 files changed.

[shorowit]

As @jmarrec said earlier, this seems like the likely culprit: #10148

[rh0dium]

@jmarrec is there anything I can do to help this issue? Happy to contribute if you point me in the right direction.

[jmarrec]

@Myoldmopar updating to CLI 2.40+ (and not relying on cmdline) is possible to remove the issue altogether, but not sure if that's a worthwhile time investment or not.

[rh0dium]

@jmarrec Wow! Thanks for digging to this and yeah I was afraid this was going to really be a can of worms. @Myoldmopar What would moving to 2.40+ look like - Can we help in any way?

[rh0dium]

FYI - I can confirm that running this one time command will solve this locally.

> docker run --platform=linux/arm64 --privileged --rm ghcr.io/norio-nomura/re-register-rosetta

Followed then by:

> docker run -it --rm --platform linux/amd64 nrel/energyplus:23.2.0 bash

root@f52c1b898f60:/# energyplus --debug-cli

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/EnergyPlus-23.2.0-7636e6b3e9-Linux-Ubuntu20.04-x86_64/Energy+.idd',

So that's a solid work around.