limit memory error on a docker file | Parsing bug

[mustafanaa]

limit memory error on a docker file | Parsing bug

[mjambon]

Good question. It should be in this week's release, watch for version 1.72.0. It should happen sometime this week or at the beginning of next week.

[mjambon]

Thank you for reporting this, it's definitely undesired behavior.

The problem goes away if I add a trailing newline character so it could be a simple parsing bug but it doesn't explain the error messages about memory:

Error while running rules: 
                    You are seeing this because the engine was killed.

                    The most common reason this happens is because it used too much memory.
                    If your repo is large (~10k files or more), you have three options:
                    1. Increase the amount of memory available to semgrep
                    2. Reduce the number of jobs semgrep runs with via `-j <jobs>`. We
                        recommend using 1 job if you are running out of memory.
                    3. Scan the repo in parts (contact us for help)

                    Otherwise, it is likely that semgrep is hitting the limit on only some
                    files. In this case, you can try to set the limit on the amount of memory
                    semgrep can use on each file with `--max-memory <memory>`. We recommend
                    lowering this to a limit 70% of the available memory. For CI runs with
                    interfile analysis, the default max-memory is 5000MB. Without, the default
                    is unlimited.

                    The last thing you can try if none of these work is to raise the stack
                    limit with `ulimit -s <limit>`.

                    If you have tried all these steps and still are seeing this error, please
                    contact us.

                       Error: semgrep-core exited with unexpected output

                       INFO:semgrep_core:Existing stack limits: Soft: 10485760, Hard: 10485760
INFO:semgrep_core:Trying to set soft limit to 1048576000
INFO:semgrep_core:Failed to set stack limit to 1048576000, 10485760. Trying again.
INFO:semgrep_core:Trying to set soft limit to 1000000000
INFO:semgrep_core:Failed to set stack limit to 1000000000, 10485760. Trying again.
INFO:semgrep_core:Trying to set soft limit to 512000000
INFO:semgrep_core:Failed to set stack limit to 512000000, 10485760. Trying again.
INFO:semgrep_core:Trying to set soft limit to 104857600
INFO:semgrep_core:Failed to set stack limit to 104857600, 10485760. Trying again.
INFO:semgrep_core:Trying to set soft limit to 52428800
INFO:semgrep_core:Failed to set stack limit to 52428800, 10485760. Trying again.
INFO:semgrep_core:Trying to set soft limit to 51200000
INFO:semgrep_core:Failed to set stack limit to 51200000, 10485760. Trying again.
INFO:semgrep_core:Trying to set soft limit to 10485760
INFO:semgrep_core:Successfully set stack limit to 10485760, 10485760
!!!This is a proprietary extension of semgrep.!!!
!!!You should not call directly this program.!!!

[mustafanaa]

Hello , @mjambon and thanks for your feedback , would this be fixed soon ?

[mjambon]

Hopefully within the next month. Ping us if this is blocking you.

[mustafanaa]

Hello @mjambon yes this issue is blocking me , can it be fixed in the upcoming days ? Thank you so much and happy weekend

[mjambon]

@mustafanaa I'll work on it next week if there's no last-minute emergency. 🤞

[mustafanaa]

Thanks , will be this on the next release ? When can i start testing it