Add PE linker switches to set and clear DYNAMIC_BASE, NX_COMPAT,
HIGH_ENTROPY_VA and TERMINAL_SERVER_AWARE without changing x86/x64
defaults. HIGH_ENTROPY_VA is limited to 64-bit PE targets and implies
DYNAMIC_BASE; clearing DYNAMIC_BASE also clears HIGH_ENTROPY_VA.
When DYNAMIC_BASE is requested for an executable, emit base relocations
and clear RELOCS_STRIPPED so Windows ASLR can relocate the image. Keep
the existing ARM64 default hardening flags.
originally, this was meant to avoid 'void' values leak
to the gen-xxx.c generators which would otherwise try
to load void from/to registers. Also catch invalid types
such as 'unsigned bool' eariler.
tccgen.c:
- expr_cond()/gexpr(): ignore VT_VOID
- gen_cast(): set float reg with static initializers to make
backends happier with invalid input
- init_putv():
improve static compound literal initializers
Also:
- ignore "missing prototype" and "might return no value"
for old style K&R functions
tcc.c, libtcc.c:
- tcc_set_output_type(): return errors from loading crt1.o
tcc.h/tccpp.c:
- fix horrible longstanding mistake with sizeof SValue.tab.
Must have place for sizeof (long double) rather than the
target's LDOUBLE_SIZE.
tccdefs.h:
- add a fake __[u]int128_t for all platforms
arm-gen.c:
- gen_cvt_ftof() must load to reg always
arm64-gen.c:
- simplify arm64_pcs() a bit
i386/x86_64-gen.c:
- do not assume char is signed (for cross compilation arm->x86)
tccelf/macho linker message:
- instead of
"undefined symbol 'X'"
now say
"unresolved reference to 'X'"
which makes it more clear that it is a _linker_error_.
tccasm.c:
- .align/p2align: use 'nop's in exec sections
tccpe.c:
- fix arm64 unwind codes (to make native set/longjmp() work)
sizeof(RUNTIME_FUNCTION) is 8 on arm64 in the first place
no need to note stack slots if we don't save any registers anyway
arm64-gen.c:
- fix long double reg-move
- fix arm64_hfa() for structs with float arrays
- gfunc_prolog(): setup stackframe eariler (simplifies unwind codes)
- new function gv_addr(RC);
win32/include/setjmp.h:
- provide correct definition for setjmo() (frameoffset = 224)
tccasm.c:
- support ".quad" with symbol & relocation
- support ".size"
- fix ". - symbol" arithmetic
win32/lib/crt1.c and win32/include/stdlib.h:
- do not write to __argc/__argv which reside in msvcrt.dll
(msvcrt.dll on arm64 does not like that, crashes on unload)
tcc.c,libtcc.c:
- new functions tcc_fopen/fclose to avoid different stdio unstances
in tcc.exe & libtcc.dll
tests & github workflow:
- add test-win32.bat to run tests with a tcc compiled by build-tcc.bat
- add msvcrt_start.c for gcc/clang to use the same runtime as tcc
the problem is that newer gcc as well as clang and cl are
linking to newer runtimes (such as UCRT) that have partially
different printf format behavior which makes tcctest fail.
the solution here is to force these compilers to link with
msvcrt.dll just like tcc.
Also, there is no gcc on arm64-win32 currently at all.
Anyway, this approach to running the github CI tests
does not require msys2. But It does rely on gnumake
as well as on some 'sh' shell though which seems to be
installed somewhere (maybe it is the one from git).
Revert "Add support to debug libtcc code"
- not fully developed experimental feature
This reverts commit 1fe3e3bff5.
This reverts commit 4768b11737.
Revert "tests: add test for x86_64 xor REX prefix bug in load()"
- AI generated nonsense test
This reverts commit d5ecb52a71.
Revert "tccpp.c: Improve integer constant overflow warning"
- Too long and confusing messages and comments for feature
with questionable benefit.
This reverts commit 085bdf8997.
riscv64-link:
- cleanup "pair pcrel lo relocations by hi address"
From fada98b1ce
tccgen.c:
- Simplify "Cast signed pointer offset to ptrdiff_t before performing arithmetic"
From 5ad52cc1ed
libtcc.c:
- Revert "tcc options: document behavior and clashes (no-op)"
a bit more information than one would like to have I think.
(why try to understand that comments plus the extra
script if one can as well just read the code itself ;)
From 234e2dd2bf
tccdefs.h:
- Revert "Move lib/va_list.c into include/tccdefs.h"
Lets not fill tccdefs.h with too much inline code
Also, -nostdlib -run is no longer supported
From fa6a6bfbbd
arm64-gen.c:
- cleanup "Implement TOK_NEG for floats natively"
Also, make it "no lvalue" in tccgen.c/x86-64-gen.c
From c39eaf10cf
lib/lib-arm64.c:
- cleanup "Remove libc dependency from lib-arm64"
using unions is much faster than some made up memcpy()
From 8c61b91de8
workflow:
- revert 'pinact for security' for readability
from 831c3fa184
tccpp.c:
- remove code that allows tcc to parse numbers incorrectly (*)
from 829c848520
tccgen.c:
- Revert "Relaxed the 'incompatible pointer type' warning a bit" (*)
from d9ec17d334.
tccrun.c:
- remove support for -nostdlib -run
for simplicity, we require "main" with tcc -run always
tccpp.c:
- Revert "Free all preprocessor memmory in case of error."
from c96f0cad61
Remove TinyAlloc->limit instead. Thus it can do also bigger
allocs. Big TokenStrings (like 200kb+ when compiling tcc)
may come from inline functions or from large initializers.
Makefile/configure:
- use --config-pie for configuring tcc output only
- use -fPIC with clang-x86_64 to avoid 32-bit relocs
libtcc.c:
- fix "tcc file.c -run" i.e. -run as last argument
i386-gen.c:
- PIC refactor
(*) sorry, but code in tcc should have a minimum of generic relevance
I tried several gdb/lldb options to debug libtcc generated code.
But gdb/lldb complained that they could not load symbols or
that module does not exist. There was also another problem. The
code itself was in memory (string) and gdb/lldb do not have
functions to acces it.
So I came up with a new api for debugging libtcc code.
First you enable debugging with: tcc_set_options(s, "-g")
Then compile the code with: tcc_compile_string_file(s, program, "<file>.c")
Then call tcc_relocate().
And finaly write the object file to disk: elf_output_obj(s, "<file>.o")
Now you can start the debugger and put an breakpoint after the
elf_output_obj() code. Then use gdb command add-symbol-file <file>.o
and from there on you can set breakpoints in the libtcc generated code.
You can also step/print variables/...
I could not find a simular function in lldb yet.
When debugging is done you remove the tcc_set_options(s, "-g").
All other code can remain because tcc_compile_string_file and
elf_output_obj do not output any file any more is debug is not set.
See also tests/libtcc_debug.c
I found some problems with a testcase from mailing list.
On x86_64 an overflow on reloc R_X86_64_32 occurred that was
not reported when using -run -ltcc.
The problem could be solved by compiling tcc with -fPIE, -pie
or --disable-static.
Makefile, configure, libtcc.c, x86_64-link.c:
- add --config-pie to configure help. Ignore -pie in libtcc.c
and check reloc overflow in x86_64-link.c
arm64-gen.c:
- Fix reading from constant like '*(int *)0x7fffb7f1280c'
elf.h, riscv64-link.c:
- fix for -run -ltcc. Ignore reloc R_RISCV_SET_ULEB128 and
R_RISCV_SUB_ULEB128 that are used in .debug_loclists section.
When tcc -run is used with input file from stdin, then run_main's
stdin (which is tcc's stdin) is already EOF after we've read it.
If the program operates exclusively on stdin, then it cannot be used.
This commit adds an option -rstdin FILE which sets run_main's stdin.
Can also be used in such case to set run_main's stdin to the tty,
using -rstdin /dev/tty (posix) or -rstdin con (windows).
The tcc options behavior is non trivial, and it's also susceptible
to ambiguities - which do exist (but currently are not painful).
- Add a script 'optclash' which detects clashes/ambiguities.
- Document the parsing behaviour and current clashes/resolutions
as comments in libtcc.c .
- include/stddef.h, tcctest.c
Revert "tests/tcctest.c: include stdint.h"
This reverts commit 8f23997ca7
We don't want tcctest.c to rely on system include files
- libtcc.c:
Revert "libtcc.c: Remove unused defines free and realloc"
To be unused is the point why they do exist
This reverts commit 2f88764100.
- tcc.c:
fix formatting of commit e73529865d
- tccpp.c:
parse_include(): print skipped include files too (with tcc -vv[v] file)
next_nomacro(): faster L"str" parsing
- tccgen.c: fix c2y if declaration:
* accept like GCC: if (int a = 0, b, c = x; c)
* accept as "TCC extension": if (int a = 0, b, c = x)
* "speak tcc" with symbol/function names
- README:
cleanup
... see tests
Also:
- tccgen.c: cleanup _Generic a little
- libtcc.c: cleanup mem-debug a little
- arm-link.c: read/write only once
- tcc.h: another fix for clang offsetof
- tccdbg.c:
* stabs: handle forward struct/enum decls
* stabs: fix anonymous struct members (must not have a .L123 name)
* avoid strncpy()
tcc.c:
- be nice to leak checkers
tcctools.c:
- remove unused TCCState params
tccrun.c:
- call bound_exit() after signals to let it free mem
tccelf.c:
- use section_add() instead of section_ptr_add() when
more appropriate
tccpp.c:
- use size_t to align tal_header naturally
- 'POINTER_SIZE' and 'PTR_SIZE' in the same source is confusing
- "char file_name[TAL_DEBUG_FILE_LEN + 1];" looks silly.
- next_nomacro(): skip UTF8 BOM at BOF
tccgen.c:
- get rid of STMT_EXPR clause on top of block
- warn with useless type like 'int;'
- move skip()'s in block() for better error line-info
- BIT_SIZE bits are meaningful only with VT_BITFIELD
(not with enums for example)
workflow/test-win32:
- build with MSVC using build-tcc.bat also
alloca.S:
- fix 'off by one' problem on win32 (must touch current page
too because the 'push %edx' at the end could touch the next page)
- must not align greater than 4 when used for struct args
(i386-gen.c:gfunc_call())
libtcc.c:
- accept -g1dwarf (dwarf output, only line info)
Tested code with:
clang -fsanitize=address,undefined,nullability -pie -fPIE -Iinclude -I. -g tcc.c -o tcc.tcc -lm -ldl -lpthread
./tcc.tcc -Iinclude -I. -b -g tcc.c -o tcc.tcc1 -lm -ldl -lpthread
Also checked on i386/x86_64 with -fsanitize=memory (others not supported).
arm-link.c: use read32le/write32le/add32le to avoid unaligned access
tcc.h i386-asm.c: fix signed left shift
lib/bcheck.c: Add _Atomic
libtcc.c: Correct MEM_DEBUG_CHECK3 to avoid unaligned access
riscv64-link.c: Fix R_RISCV_SET16
tccpp.c: Align tal_header_t to avoid unaligned access
tccgen.c x86_64-gen.c: avoid use of uninitialized value
There are still warnings reported:
tccgen.c:4031:13: runtime error: member access within null pointer of type 'TCCState' (aka 'struct TCCState')
tccelf.c:321:22: runtime error: applying zero offset to null pointer
tccelf.c:1132:23: runtime error: applying non-zero offset 169184 to null pointer
A lot of left shift of negative value warnings.
I ignored these for the moment.
Also the -run option does no work well with -fsanitize. It gets confused
because it does not detect that the generated code in memory is used
without -fsanitize option.
There are a lot more -fsanitize options. I did not find serious problems
with them.
workflows/build.yml:
- win32/64: install mingw32/64 gcc on msys (because the default
gcc installed elsewhere seems to use ucrt, producing incompatible
fp printf formats.)
tccgen.c:
- cleanup funcs: save any lvalues from return expressions. Also use
get_temp_local_var() which however was causing a problem on i386
because its gfunc_call() removes the arguments from vstack and by
that defeats the 'in-use' tracking of get_temp_local_var(). Fixed by:
i386/arm/arm64/x86_64-gen.c:
- in gfunc_call(): save_regs before anything else, fixes
problems seen in arm64/i386
tccpp.c:
- allow arm asm specific use of '#' in macros
libtcc.c:
- organize -M options, and:
tccpe.c:
- move the subsystem option parsing from libtcc.c
tccelf.c:
- improved error handling in tcc_load_ldscript()
lib/atomic.S:
- TCC_TARGET_... not defined when building the lib
- endbrNN security feature not supported by TCC
tests/tests2/136_atomic_gcc_style.c:
- never use standard assert() in tests
tcc_set_str() already calls tcc_free() to free the memory of the stringpointer in s->outfile.
With the additional tcc_free(), the memory would be freed twice.
--
Regards ... Detlef
cleanup libtcc.c:tcc_set_linker()
cleanup tccelf.c:tcc_load_ldscript()
Also
- tccrun.c, tccelf.c:relocate_syms():
with tcc -run -nostdlib, do resolve but only from explicitly
on the command-line given libraries.
- tccgen.c: optimize UMOD x % c -> x & (c-1) for c = 2^n
- tcc-doc.texi: cleanup
- tcc.h, tccpp.c, libtcc.c: add 'size' arg to pstrncpy()
Also reorder functions in libtcc.c a bit.
9 files changed, 556 insertions(+), 617 deletions(-)
Oviously what the patch wants is... on Windows, with
"tcc c:/dir/xyz.dll -run file"... pass the absolute
path of the dll to LoadLibrary() which can make sense
in situations.
Other changes in the patch to other platfurms seem to
have no effect. This reverts 52a9a541b0
except 2 lines in tccpe.c.
Also revert _Float16 patch a06c608625
As long as tcc does not really handle _Float16, we can just define
it in tccdefs.h.
Also move uint128_t to tccdefs.h for same reason.
Update github action (might fix random arm64 crashes)
libtcc.c:
- free 'elfint' string
- acceot -O and -Os
- accept -gstabs (to override dwarf when default)
- better -Wp,...
tccpp.c:
- #line cleanup
also warn with "extra tokens after directive"
tccgen.c & xxx_gen.c:
- force CPU flags to register earlier
tccelf.c:
- tcc_load_object: align size only for code sections
data/bss objects are always put with their specfic type align
(in decl_initializer_alloc())
x86/64 doesn't need aligned code
from c6afdff7ab
tccpe.c:
- enable dllimport for "_imp__<sym>" also from assembler
x86_64-gen.c & lib/libtcc1.c:
- simpler fneg without libtcc1 reference
tests2/134_double_to_signed.c:
- a tcc compiled by msvc won't pass this test
TCC treats -nostdlib and -Wl,-nostdlib as equivalent, but on other compilers
which call a discrete linker, -nostdlib behaves as on tcc (not adding
startfiles/libc/endfiles) by modifying the ld command line, but -Wl,-nostdlib
adds -nostdlib to the ld cmdline, which stops the linker searching the default
directories for libraries.
just clear s1->static_link before loading msvcrt etc.
Also:
- configure: get cc_version/name when making a cross compiler too
- configure: fix android triplets
- Makefile: remove CONFIG_TCC_CROSS, check TCC_TARGET_xxx instead
- libtcc.c: parse some linker option arguments more correctly
- tccelf.c: fix a versym problem with clang on android
- lib/Makefile, build-tcc.bat: bcheck.c now includes config.h
Put former NATIVE_DEFINES into config.h. Such tcc can be run
and tested directly from source more easily, like for example:
tcc -run tcc.c -B. -run test.c
Also:
- tccelf.c: cleanup
- tccpp.c: avoid stupid clang warning
- configure: reduce -Wno- switches
- tcc.h: inline wait/post_sem()
- tccpe.c: simplify import (assume STT_NOTYPE is function)
with -Wl,-oformat=binary, executable code should come first.
(for linux kernel image for example)
Also:
- simplify RELRO sections: create them as readonly, but add
SHF_WRITE flag later when needed (i.e. relocations do exist)
- tcc.h etc: exclude eh_frames on non-elf platforms
- tccelf.c:tcc_load_object_file(): don't load debug sections when
linking without -g (special dwarf case in relocate_section()
wont work when dwlo/hi were not initialized).
- tcc.c: avoid loop if something fails (ret < 0) without message
(while failing without message should not happen either)
- tccelf.c:tcc_load_alacarte: give message
- tccpp.c: treat '# 123xyz' in asm file as comment
- lib/Makefile: cleanup
- libtcc.c: tcc_add_library(): fallback to try filename as is
(also remove tcc_add_library_err())
patch originally made to prove correctness (comparing stages)
with tinycc compiling gcc 2.95.3 which would assign registers
differently (but still correctly) when compiled with tcc without
this option).
Also: fixes get_temp_local_var() which on 32-bit systems
happened to return a temporary location that was still
in use because its offset was changed on the vstack
(incremented by four in gv() to load the second register
of a long long).
Also: optimize vrot-t/b (slightly) by using one memmove
instead of moving elements one by one in a loop.
- output correct line number with "error: duplicate case value"
- libtcc.c:error1(): support specific line numbers with "%i:"
tcc_error("%i:message ...", line_num, ...);
Also:
- simplify signed/unsigned switch compare
- optimize implicit case ranges such as
case 1: case 2: case 3: ...
- simplify llong constant propagation in gen_opic()
- rename Sym.ncl to Sym.cleanup_func
- tccgen.c: cleanup switch data etc. after errors (*)
- tccpe.c: faster get_dllexports (*)
- tccpe.c: support -Wl,-e[ntry]=... (*)
- libtcc.c: win32: use ANSI functions (GetModuleFileNameA etc.)
- tccrun.c: be nice to tcc-0.9.26 ("struct/enum already defined")
- tccpp.c: be nice to tcc-0.9.27's va_start/end macros
(*) suggested by Robert Schlicht
https://lists.gnu.org/archive/html/tinycc-devel/2024-03/msg00012.html
- LIBTCCAPI int tcc_set_backtrace_func(void *ud, ...)
accept opaque user data pointer,
- tcc -vv -run... : show section info
- use memalign() to allocate runtime memory
- printline_/dwarf : pass output to parent function
- tccpe.c : fix -nostdlib -run
- --config-backtrace=no : make it work again
- abort with notice when tcc_relocate() is called with the
former two-step method
- support backtrace & bcheck not only with tcc_run() but also
for directly called functions from tcc_get_symbol(); enable
witn 'tcc_set_options("-bt/-b");'
- move struct rt_context and debug sections into compiled code
for TCC_OUTPUT_MEMORY also
- protect access (g_rc) with semaphore
Also:
- add armv7/aarch4/riscv64 github tests (qemu emulated)
- win32/build-tcc.bat: build cross compiler only with -x
- remove TOK_NOSUBST, mark the token itself instead
- get_tok_str(); mask out SYM_FIELD & update uses
- next(): optimize (~5% faster with tcc -E)
- tok_flags: remove some redundancy
- parse_define(): do not remove spaces around '##' and after '#'
and mark macros with '##' as MACRO_JOIN to avoid unnecessary
call to macro_twosharps(mstr):
- next_nomacro(): removed, next_nomacro1(): renamed to next_nomacro()
- next_argstream(): cleanup & new function peek_file()
- macro_subst_tok(): handle special macros (__DATE__ etc.)
like normal macros if they are #defined
- -DPP_DEBUG : more structured output
- pp_error(): better preprocessor expression error message
- tcctok.h: sort basic keywords (somehow)
- testspp/Makefile: generate .expect with 'make testspp.##+'
- tcc.c: tcc -E -o file : put unixy LFs also on windows
removed second argument for tcc_relocate(s). previous
'TCC_RELOCATE_AUTO' is now default and only behavior.
Rationale:
In the past, the option to compile into memory provided by the
user was introduced because only one TCCState could exist at a time.
This is no longer a limitation. As such it is also possible now to
keep any number of compiled code snippets around together with their
state in order to run them as needed.
- Also
- LIBTCCAPI tcc_get_error_func/opaque() removed
- tccrun/SELINUX: switch rx/rw mappings such that rx comes first
(risc64-link.c:relocate_plt() does not like got < plt)
- tcc_relocate_ex(): free local symbols and obsolete sections
to reduce memory after tcc_relocate()
When using libtcc it's reasonable to be able to use the application's
memory allocator for all allocations, including tcc_new(), and including
#define MEM_DEBUG
Ideally the allocator would be stored in the TCCState, like TCCErrorFunc.
That would imply a new API tcc_new_with_allocator(), but more importantly
would require all uses of tcc_malloc(x) to be changed to s->tcc_malloc(x).
That's a non-starter in my book.
Instead I refactored the memory management code so that all allocations
flow through tcc_realloc(). Which simply calls a function pointer, the
default value of which is the previous tcc_realloc().
It then becomess trivial to install a new allocator with the new function:
LIBTCCAPI void tcc_set_realloc(TCCReallocFunc realloc);
The resulting code adds the trivial cost of an additional function call
per allocation/free. It also doesn't distinguish between malloc failure
and realloc failure, but since both just fprintf then exit() that seems
unimportant to me.
On the plus side the preprocessor magic is much more clear. The diffs
don't hightlight that, but take a look at the result to see if you agree.
All tests passed on my x86 linux box.
tccpp.c:
- revert "Preprocessor fix + new testcase"
Fix was not a fix and nobody could understand the test.
This reverts 6379f2ee76
- better fix and add new test (pp/18.c)
tccgen.c:
- remove global variables 'in_sizeof', 'constant_p'
- rework comma expression (gexpr())
- merge func/data 'alias_target' codes
(See 08c777053c)
- move call to do_Static_assert()
- better error: "expression expected before '%s'"
- fix "statement after label"
- remove unnecessary second parameter to block()
- remove unnecessary call to decl()
- revert changes to old C89 test file
See 7f0a28f6ca
tccelf.c:
- rework "...make undefined global symbol STT_NOTYPE"
(See f44060f8fc)
- move tccelf_add_crtbegin() from libtcc.c
tcctest:
- unfix K&R fix (keep old look of K&R functions)
tccrun.c:
- exit(0) returns 0
libtcc.c:
- move #defines for -dumpmachine
- more explicit error "file not found"
(as opposed to error while loading file)
tccpe.c, x86_64-gen.c, i386-asm.c, tccasm.c:
- use R_X86_64_PLT32 for functions on x86_64-win32
tccdefs.h
- empty #defines for _Nonnull, __has_builtin(), etc.
configure:
- Simpler "macOS .dylib ... VERSION letters."
(See 6b967b1285)
Makefile:
- macOS version also
- add cross searchpaths for packages
build.yml:
- disable codesign on macos-11 (doesn't seem to work)
