elliotchance/c2go

⚖️ A tool for transpiling C to Go.

c2go's Issues

Only export interfaces from ast package

This would be a follow on from #45. If we keep the render functionality in the ast package we might get into messy dependencies between other packages when we split out other parts (like the type resolution) into another package. Also ast would just become extremely heavy with logic.

This is open for discussion, but my initial thought is to eventually only have interfaces for each of the AST nodes. The ast packages would still be responsible to processing the clang AST and maintaining the data structures internally. We would only need the Parse method and each of the interfaces to be exported.

Also, the Children attribute on all nodes would not need to be there. Instead we could provide much more meaningful property names. For example, rather than having 4 or 5 children for a ForStmt it would make much more sense to expose through the interface:

type ForStmt interface {
    // ...
    Init() Node
    Conditional() Node
    Step() Node
    Body() Node
}

Thoughts? @bongo227

Add a --version flag to c2go

Basically, add a --version flag to see what version of c2go is installed in this moment.

Only some modes are supported by fopen()

Add a license

MIT

Provide new code samples (if you are new, start here)

http://www.cplusplus.com/reference/ is a great source for code samples since they have examples for most functions, like: http://www.cplusplus.com/reference/cmath/cos/

Here's how to contribute:

1. Have the project cloned and working (run go test -tags=integration).
2. Put one of the samples (any sample that is a complete program and has at least one printf() will do) in the tests/ directory.
3. Try to run it with: go test -tags=integration -run TestIntegrationScripts/tests/ctype/isalnum.c
4. Have a go at trying to fix any errors - if you can't fix the errors, don't worry! Even opening a PR with a broken build is helpful. The PR also acts as a great place to get feedback, ask for help, or even get someone else to help you out.

It is best to keep PRs as small as possible, so even providing a PR with the code for single function would be fantastic.

Happy coding!

Some platform structs are ignored

There are some structs that have been ignored for various reasons. The code to ignore them should be removed. Search the codebase for "issues/85"

Allow output as JSON

There is already an option for -print-ast but it would be nice to have a JSON only output so the AST can be ingested and analysed by other programs. This functionality already exists, but is commented out.

Help complete the "What Is Supported?" table

In the README there is a table of what functions are currently supported.

At the moment it only includes the few functions that are known to be supported. It does not include the vast majority of C functions that are unsupported or untested. With a more comprehensive table we can get a better understanding of the progress of the project and what needs to be done.

Also, the table does not include non-library items, such as which language constructs (for loops, structs, etc) are supported or not. This would also be very helpful.

Support for stdio.h

At the moment it only works for a handful of print/scan functions to get basic integrations tests working, but there is a lot more in stdio.h to be implemented.

File IO

#include <stdio.h>

int main()
{
	printf("Hello World\n");

	FILE *f = fopen("test.txt", "w");
	if (f == NULL)
	{
		printf("Error :(\n");
		return 0;
	}

	fprintf(f, "Hello World\n");

	fclose(f);
}

Converts to:

func main() {
	fmt.Printf("Hello World\n")
	var f *int64 = fopen("test.txt", "w")

	if f == nil {
		fmt.Printf("Error :(\n")
		return

	}

	fprintf(f, "Hello World\n")
	fclose(f)

}

Nice try, but it needs a tiny bit more work ;)

Ability to evaluate constant expressions at compile time

Some header files contain constant expressions (usually from macros) that the compiler is able to evaluate at runtime. An example of this is in ctype.h: https://code.woboq.org/linux/include/ctype.h.html#46

There is a macro which is used by the enum values:

#define _ISbit(bit)	((bit) < 8 ? ((1 << (bit)) << 8) : ((1 << (bit)) >> 8))

MemberExpr does not know member types

We currently do not remember the types of fields in a struct. There is a few hacks in there to get around it, but we really need a proper solution.

Also, struct.c only really works by coincidence.

Speed up integration tests

It's nice to have one C file per function, but where there is little change in syntax between the files (like testing all the math.h functions) this just means the tests take a long time.

We should consider reducing these into a single test file, like tests/math.c which will make testing much faster.

Fix the return value of Render()

At the moment all Render() methods return []string. This is because it was a direct translation from the original Python, however this can be improved.

The Render() method really returns two values:

1. The rendered expression as a string, like a + (2 * b)
2. The C type of that expression, like double.

Use the go AST instead of building the source manually

Example addition for stdbool.h / _Bool panic

Panics when finding _Bool type: panic: '_Bool'

stdbool.c

#include <stdio.h>
#include <stdbool.h>

int main() {
  
  bool trueBool = true;
  bool falseBool = false;
    
  if(trueBool) {
    printf("true bool\n");
  }
  
  if(!falseBool) {
    printf("reversed false bool\n");
  }
  
}

Add support for enums

Refactor type resolution into separate package

As mentioned on reviewable: "This should also make it easier to handle more complex and state-oriented type resolutions like function pointers." - @elliotchance.

Fix up the README

There are still mentions of Python. And now there is a wiki page for the progress: https://github.com/elliotchance/c2go/wiki/Project-Progress

Support for clang 3.5

Clang 3.5 was left out of the CI build process because it didn't work like the other versions of clang. I'm not sure why, and I didn't want to spend too much time investigating at the time.

Now we should revisit it, and fix it.

panic on empty for-loop conditions

for (;;) is legal C, but currently causes a panic. For example, in the following program, c2go attempts to treat the print statement as the initialization part of the loop condition.

main(int argc, char **argv)
{
	for (;;)
		print("y\n");
}

Convert ast2json.py to Go

ast2json.py is a standalone tool to injects the output from the clang AST, parse it with a bunch of regexes and produce a JSON output.

It currently works like this:

clang -Xclang -ast-dump -fsyntax-only myfile.c | python ast2json.py

It works with stdin and stdout, so the same could be achieved with a compiled Go application.

Split ast_tests.go into multiple files

It's becoming a pain to scroll through 1000+ lines of ast tests and there will be a lot more to come.

The logic for running the AST tests can be extracted into a function and we can create one test per file, like while_stmt_test.go:

var tests = map[string]Node{
	`0x7fa1478273a0 <line:7:4, line:11:4>`: &WhileStmt{
		Address:  "0x7fa1478273a0",
		Position: "line:7:4, line:11:4",
		Children: []interface{}{},
	}
}

func TestWhileStmt(t *testing.T) {
	runASTTests(t, "WhileStmt", tests)
}

Support freopen()

Is currently not supported.

Command-line flags

It would be nice to have some command-line flags to support multiple operations:

• -transpile: pretty much the current behavior, generates a .go source code for each C file passed as argument
• -run: transpile .c source code into Go and perform a run directly, just like done by the "go run" command in https://github.com/golang/go/blob/master/src/cmd/go/internal/run/run.go.
• -build: transpile C code myfile.c to Go and build it to generate a static binary called myfile, would also accept a -out parameter to be able to customize the binary name.

go test & run-tests.sh failing - expected 'package', found 'EOF'

I clone the repo, ran go get and then go test, but unfortunately it fails with a panic:

shell@shell-virtual-machine ~/g/s/g/e/c2go> go test
--- FAIL: TestIntegrationScripts (0.10s)
panic: '' [recovered]
	panic: ''

goroutine 10 [running]:
testing.tRunner.func1(0xc42006f380)
	/usr/local/go/src/testing/testing.go:622 +0x29d
panic(0x547960, 0xc420383a00)
	/usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/elliotchance/c2go.resolveType(0x0, 0x0, 0x57e87e, 0x8)
	/home/shell/gocode/src/github.com/elliotchance/c2go/resolve.go:167 +0xc4f
github.com/elliotchance/c2go.(*TypedefDecl).RenderLine(0xc420372900, 0xc4201c88c0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/typedef_decl.go:65 +0x162
github.com/elliotchance/c2go.Render(0xc4201c88c0, 0x551f60, 0xc420372900, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/common.go:48 +0x102
github.com/elliotchance/c2go.(*TranslationUnitDecl).RenderLine(0xc42028e2a0, 0xc4201c88c0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/translation_unit_decl.go:21 +0xa5
github.com/elliotchance/c2go.Render(0xc4201c88c0, 0x551f00, 0xc42028e2a0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/common.go:48 +0x102
github.com/elliotchance/c2go.Start(0xc4203eaf78, 0x2, 0x2, 0x21, 0x40)
	/home/shell/gocode/src/github.com/elliotchance/c2go/main.go:156 +0x38b
github.com/elliotchance/c2go.TestIntegrationScripts(0xc42006f380)
	/home/shell/gocode/src/github.com/elliotchance/c2go/main_test.go:20 +0xbd
testing.tRunner(0xc42006f380, 0x58b3d0)
	/usr/local/go/src/testing/testing.go:657 +0x96
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:697 +0x2ca
exit status 2
FAIL	github.com/elliotchance/c2go	0.111s

run-tests.sh also seems to produce an empty panic:

shell@shell-virtual-machine ~/g/s/g/e/c2go> ./run-tests.sh 
CLANG_BIN=clang
CLANG_VERSION=clang version 3.8.1-12ubuntu1 (tags/RELEASE_381/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

tests/misc/user-input.c
panic: ''

goroutine 1 [running]:
main.resolveType(0x0, 0x0, 0x51a90a, 0x8)
	/home/shell/gocode/src/github.com/elliotchance/c2go/resolve.go:167 +0xc4f
main.(*TypedefDecl).RenderLine(0xc4200d4d80, 0xc420176000, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/typedef_decl.go:65 +0x162
main.Render(0xc420176000, 0x4f8ca0, 0xc4200d4d80, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/common.go:48 +0x102
main.(*TranslationUnitDecl).RenderLine(0xc4200a84b0, 0xc420176000, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/translation_unit_decl.go:21 +0xa5
main.Render(0xc420176000, 0x4f8c40, 0xc4200a84b0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/common.go:48 +0x102
main.Start(0xc42000a440, 0x2, 0x2, 0x0, 0x0)
	/home/shell/gocode/src/github.com/elliotchance/c2go/main.go:156 +0x38b
main.main()
	/home/shell/gocode/src/github.com/elliotchance/c2go/main.go:171 +0x49
can't load package: package github.com/elliotchance/c2go/build: 
main.go:1:1: expected 'package', found 'EOF'
=== out.go
cat: build/main.go: No such file or directory

Add "go fmt" to travis

This is to make sure the codebase is always formatted correctly. The build should fail if go fmt finds files to format.

System arguments are fixed variable names

The CLI arguments are currently fixed variable names (argv and argc). They do not read the definition of main().

Very useful

I reckon this would be useful for machine learning c packages.
They mostly have no other dependencies like file systems or databases but are designed to be called and return results.

For example OCR in tesseract etc.

I am trying to build an OCR system and so this repo might be a good approach for me.

Could you comment on the soundness if my approach ?

Transpile special character literals

A character literal is a single character expression like 'a'. Character literals are currently handled in transpileCharacterLiteral (cryptic name, right?) but it does not handle special cases like newlines, tabs or multibyte characters.

Unions are not supported

Compiling multiple files

At the moment all the preprocessed code is output into a single Go file. If there were multiple files process you would get lots of duplicate symbols in the same package.

Remove run-tests.sh

At the moment we run TestIntegrationScripts and run-tests.sh. These could really be done at the same time tin go test.

Project chat room

It would be nice to have a chatroom on IRC, gitter.im or Slack so users can easily get in touch with developers.

I would vote for gitter.im, since that's quite well integrated with github repos out of the box.

Unused variables are an error in Go

Variables that are declared but not used are considered an error in Go.

Correctly handle the exit code returned from main()

The main function in Go does not return a value. c2go will discard the return value so the program will always finish with a successful exit code.

We need to recognise the return in the main function and call the system exit so that the correct status is passed back to the calling process.

Search for "issues/79" in the codebase.

Jsonnet

Thanks for this project.

I am wondering if it's nuts to try to convert this :
https://github.com/google/jsonnet

I am not a c programmer but can hack my way through it so it's hard for me to judge if a library as complex as jsonnet is too big a leap.

The reason i want a golang version is because kubernetes uses this to driver configuration codegen.

https://github.com/kubernetes/kubernetes-anywhere

syntax error: unexpected short

When running ./run-tests.sh I get:

~/clones/c2go% ./run-tests.sh 
argv.c
# command-line-arguments
./out.go:12: syntax error: unexpected short after top level declaration
./out.go:16: syntax error: unexpected long after top level declaration
./out.go:22: syntax error: unexpected short after top level declaration
./out.go:24: syntax error: unexpected short after top level declaration
./out.go:26: syntax error: unexpected int after top level declaration
./out.go:30: syntax error: unexpected long after top level declaration
./out.go:32: syntax error: unexpected long after top level declaration
./out.go:36: syntax error: unexpected long after top level declaration
./out.go:40: syntax error: unexpected long after top level declaration
./out.go:42: syntax error: unexpected long after top level declaration
./out.go:42: too many errors
1,3d0
< Number of command line arguments passed: 3
< 2. Command line argument passed is some
< 3. Command line argument passed is args

This is after changing "python" to "python2" in run-tests.sh.

I'm on 64-bit Arch Linux with:

• Python 2.7.13
• Python 3.6.
• Go 1.8
• clang 3.9.1
• python-libclang 3.8.2

SQLite

I wish I could contribute to this project, but unfortunately my python skills are beyond non-existent. :( Have you considered moving this to Go instead?

The bane of my existence these days is SQLite, because it requires cgo. For giggles I thought I would run this against it to see what I got. Here are the results:

There is no regex for 'ParenType'.
I will print out all the lines so a regex can be created:

 0x7fa205805b00 'int (void *, int, char **, char **)' sugar
 0x7fa20580b670 'void (void)' sugar
 0x7fa20580f660 'void (void *)' sugar
 0x7fa20584cdc0 'void (const Fts5ExtensionApi *, Fts5Context *, sqlite3_context *, int, sqlite3_value **)' sugar
 0x7fa205872be0 'void (int)' sugar
 0x7fa206058270 'int (int, const void *, UnpackedRecord *)' sugar
 0x7fa205815e30 'int (void *, int, const char *, const char *, const char *, const char *)' sugar
 0x7fa205098b40 'const sqlite3_io_methods *(const char *, unixFile *)' sugar
 0x7fa20690d4e0 'int (SortSubtask *, int *, const void *, int, const void *, int)' sugar
 0x7fa205c496c0 'int (sqlite3 *, char **, const sqlite3_api_routines *)' sugar
   `-ParenType 0x7fa206091310 'void (void *, int, const char *)' sugar
   `-ParenType 0x7fa20580fc10 'int (int)' sugar
   `-ParenType 0x7fa20580b670 'void (void)' sugar

Traceback (most recent call last):
  File "c2go.py", line 568, in <module>
    raise e
ValueError: No JSON object could be decoded

While that is probably not very helpful to you, I thought that giving you more examples might be of at least some help.

If you move this project to Go, or decide to set up a patreon.com page, please let me know.

I am willing to throw some money at converting SQLite to pure Go! :)

Goodluck

Example addition for switch statements

Tried to transpile a simple program with a switch statement and it failed with the following error:
panic: Unknown node type: 'SwitchStmt 0x55c465236c38 <line:10:3, line:23:3>'

switch.c

#include <stdio.h>

int main() {
  
  int num;
  printf("Give me either an 1 or a 2: ");
  scanf("%d", &num);

  switch(num) {
    case 1:
      printf("1\n");
      break;
    case 2:
      printf("2\n");
      break;
    default:
      printf("Invalid input.\n");
      break;
  }
}

Some functions are ignored because they are too much trouble

The title pretty much says it all. You can find them in the code base by searching for "issues/78"

No Regex found error

When running the tests, I get:

There is no regex for 'FunctionProtoType'.
I will print out all the lines so a regex can be created:

 0x29b1190 '__ssize_t (void *, char *, size_t)' cdecl
 0x29b1430 '__ssize_t (void *, const char *, size_t)' cdecl
 0x29b16d0 'int (void *, __off64_t *, int)' cdecl
 0x29b1810 'int (void *)' cdecl

Failing to run c2go on kilo.c

While trying out this project I tried and fail to run it against kilo.c

c2go kilo.c 
panic: could not match regexp '(?P<address>[0-9a-fx]+) <(?P<position>.*)>(?P<position2> [^ ]+)?(?P<referenced> referenced)? (?P<name>\w+?) '(?P<type>.+?)'' with string 'FieldDecl 0x21391d8 <line:75:2, col:15> col:2 'unsigned int''

goroutine 1 [running]:
main.groupsFromRegex(0x5228cf, 0x6f, 0xc4203e35aa, 0x3c, 0x1)
	/home/yml/gopath/src/github.com/elliotchance/c2go/ast.go:150 +0x392
main.parseFieldDecl(0xc4203e35aa, 0x3c, 0x51aa8a)
	/home/yml/gopath/src/github.com/elliotchance/c2go/field_decl.go:26 +0x4e
main.Parse(0xc4203e35aa, 0x3c, 0x42, 0xc4200af6e0)
	/home/yml/gopath/src/github.com/elliotchance/c2go/ast.go:69 +0xb9c
main.convertLinesToNodes(0xc4200ba000, 0x2631, 0x2631, 0xc4200ba000, 0x2631, 0x2631)
	/home/yml/gopath/src/github.com/elliotchance/c2go/main.go:38 +0x1d8
main.Start(0xc420076080, 0x2, 0x2, 0x0, 0x0)
	/home/yml/gopath/src/github.com/elliotchance/c2go/main.go:143 +0x289
main.main()
	/home/yml/gopath/src/github.com/elliotchance/c2go/main.go:171 +0x49

I have close to no idea what I am doing and running c2go on this kind of project is may be out of scope. Thank you for opening up this project.

TODO:

• <termios.h>
• <stdlib.h>
• <stdio.h>
• <errno.h>
• <string.h>
• <stdlib.h>
• <ctype.h>
• <sys/types.h>
• <sys/ioctl.h>
• <sys/time.h>
• <unistd.h>
• <stdarg.h>
• <fcntl.h>

Compile knightstour.c

This was an issue raise originally by @carstenhag with #27.

The original ticket was fixed but the example program was very good. This issue will outline what we need to do to get knightstour.c working.

• stdio.h (#61)
• stdlib.h
• stdbool.h (#65)
• string.h
• math.h (#9)
• Better struct handling for coord and extCoord (#49)
• qsort()
• Memory allocationcalloc() and free().
• sizeof() (#102)

The name of a variable or field cannot be "type"

type is a reserved word in Go so we not only have to replace it at the source of the declaration but also any time a variable or struct field is referenced.

Could not match regexp - FieldDecl - unsigned short

Machine: Macbook, OS: 64bit Mac OS X 10.12.3 16D32 , Kernel: x86_64 Darwin 16.4.0 (different machine than the other issue)
knightstour.c Gist

It consists of a few basic input/output stuff, recursive methods, simple math functions and memory allocation.

Hi, I tried to transpile a C program I wrote but unfortunately c2go seems to have a problem with unsigned shorts, yet I don't use that data type at all in the program.

~ λ c2go knightstour.c
panic: could not match regexp '(?P<address>[0-9a-fx]+) <(?P<position>.*)>(?P<position2> [^ ]+)?(?P<referenced> referenced)? (?P<name>\w+?) '(?P<type>.+?)'' with string 'FieldDecl 0x7f9bc9083d00 <line:91:5, line:97:8> line:91:5 'unsigned short''

goroutine 1 [running]:
main.groupsFromRegex(0x1122b7a, 0x6f, 0xc420298d34, 0x4a, 0x1)
	/Users/chag0709/go/src/github.com/elliotchance/c2go/ast.go:150 +0x392
main.parseFieldDecl(0xc420298d34, 0x4a, 0x111b029)
	/Users/chag0709/go/src/github.com/elliotchance/c2go/field_decl.go:26 +0x4e
main.Parse(0xc420298d34, 0x4a, 0x4e, 0xc4200841e0)
	/Users/chag0709/go/src/github.com/elliotchance/c2go/ast.go:69 +0xb9c
main.convertLinesToNodes(0xc4202fe000, 0x16f9, 0x16f9, 0xc4202fe000, 0x16f9, 0x16f9)
	/Users/chag0709/go/src/github.com/elliotchance/c2go/main.go:38 +0x1d8
main.Start(0xc42007c060, 0x2, 0x2, 0x0, 0x0)
	/Users/chag0709/go/src/github.com/elliotchance/c2go/main.go:143 +0x289
main.main()
	/Users/chag0709/go/src/github.com/elliotchance/c2go/main.go:171 +0x49

Predefined expressions are not evaluated

Predefined expressions are compiler macros that evaluate to literals at compile time. For example __PRETTY_FUNCTION__.

Search for "issues/81" in the code base.

clang.cindex.LibclangError: libclang.so

I get the following error after installing llvm, clang, libclang-dev via package manager and installing the python bindings from the link in the readme (tar file, then python setup.py install):

clang.cindex.LibclangError: libclang.so: cannot open shared object file: No such file or directory. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().

Which distro did you use?
Which packages did you install?
How did you install the python bindings?

Add multiple versions of clang to Travis CI

The c2go build system should include all versions of clang 3+ to be tested against mac and windows.

Allow arguments to be passed by reference when applying a substitution

Some C functions take a buffer argument, or return the result or both. For example:

char* tmpnam(char*) -> noarch.Tmpnam

If the first argument is not NULL the result is written there. This means that you can use this function without actually assigning it's return value to a variable. These would be the same:

char *f = tmpnam(NULL);

char f[256];
tmpnam(f);