Comments (23)
I was curious how the combination of Embedded Swift and wasm would change this large file size issue. I was surprised to find that when I built the one shown in Wasm I/O at hand, it was an astonishingly small 101k bytes! This definitely indicates that swiftwasm can be adopted for some development, including deployment to wasm.
102810 swift-audio.wasm
Apple Example: https://github.com/apple/swift-for-wasm-examples
Related video: https://www.youtube.com/watch?v=6yxPavqB144
from swift.
Now, we're focussing on making produced binary work properly. But I think binary size is one of the most important things too.
Fat stdlib
As far as I investigated, Swift stdlib is so fat that weights about 8.8MB because it depends on ICU and libc. Other platform projects are facing on same problem too.
Fortunately, Swift6 will work on supporting more platforms, and core team know this problem. If they decide to implement a subset of stdlib to be more small, it can be helpful.
LTO approach
Most of stdlib parts are unused, so if linker omits unused code, the produced binary will be more small.
Here is a Swift LTO talk at LLVM meeting.
This presentation shows that effective LTO reduces 20% of binary size. I've not tried this method yet and I don't know whether this works as well for wasm target, but in theory it can.
Swift has many dynamic language feature, like dynamic casting, dynamic dispatch or etc.., so my concerns with this approach is that linker can't know what protocol
are used.
For example,
- ModuleA.swift has
protocol P
and conformance ofP
forString
andString.foo
method.
public protocol P {
func foo()
}
extension String: P {
func foo() {}
}
public func useP<T>(_ value: T) {
(value as? P)?.foo()
}
- main.swift import
ModuleA
but not use it.
import ModuleA
struct S: P {}
useP(S())
In this case, ModuleA
's conformance and String.foo
are included in produced binary because linker can't know whether the main.swift uses String: P
conformance or not. (conformance metadata are always marked as llvm.used
to reference them sequentially)
So, to achieve more reduction, we need to consider new optimization approach, like WMO or something LTO plugin that tells linker which protocols are used and omits unused conformances and functions.
from swift.
After running wasm-strip
, it reduces to 7.5 Mb
Next, after running wasm-opt -Os
it gets to 3.4 Mb
Still quite big, as with AssemblyScript
, Rust
, and TinyGo
I was able to generate useful wasm files of just 200...3500 bytes. See https://github.com/wasm3/wasm3-arduino/tree/master/examples_pio/Wasm_Advanced/wasm_apps
from swift.
Nice!
I just got it running!! The file is now much smaller as you pointed (9Mb), and takes much less time to execute.
I also published a new version of Wasienv that uses the latest release:
wasienv install-swift
wasiswiftc example.swift -o example.wasm
π
Timings
While trying things out, I've been doing an analysis of the example.wasm
execution in different runtimes.
I just compared wasmer and wasmtime for running a generated file, and Wasmer timings are much better (about 10x faster runtime speed once the file is compiled)
Wasmtime: 1s to execute
β wasienv git:(master) β time wasmtime example.wasm
Hello, WASI!
wasmtime example.wasm 0.97s user 0.10s system 98% cpu 1.087 total
Wasmer: .2s to execute (5x faster)
β wasienv git:(master) β time wasmer example.wasm
Hello, WASI!
wasmer example.wasm 0.12s user 0.08s system 96% cpu 0.215 total
from swift.
I've posted an update about ICU here recently: #2411 (comment). Basically, we should phrase this as "removal of Unicode tables" instead of "removal of ICU". Even though ICU was removed as a dependency, Unicode tables were copied from it to stdlib itself, that shouldn't make a big difference in binary size if any.
As for build flags, there's ongoing work on swiftlang#42366, which may be relevant, but I haven't done any measurements with that myself.
WRT custom allocators and WASI-less builds, we could link with uSwift instead of stdlib, but we need to make #4374 fully work, and uSwift itself is super barebones and doesn't even support generics. This option is going to take most time and effort to implement when compared to other options, in my opinion.
from swift.
Ok, thanks so much for your continued work on this @kateinoigakukun. You are doing heroic work. γ©γγγγγγ¨γγγγγΎγγ
from swift.
I can confirm that running wasm-strip
from wabt and wasm-opt -Os
from binaryen (in this order, wasm-opt
seems to choke on raw binaries produced by swift build
without having wasm-strip
run on them at first) reduced an 11M binary to 4.9M. Interestingly enough the same code built in release mode with swift build -c release
was only reduced to 5.1M π€
Both became 1.5M when zipped, that's not great, but not terrible either, almost order of magnitude better than the raw binary π
from swift.
@MaxDesiatov wrote up a great post about using the upcoming Embedded feature of Swift to build small binaries: https://forums.swift.org/t/some-feedback-from-my-short-experience-with-swiftwasm/69605/5
from swift.
I tried to use wasmer instead of wasmtime, it saved 13min for test execution! It's really good news!
#451
from swift.
@MaxDesiatov How could one assist in reducing the binary size?
Is it possible to use uSwift with swiftwasm? If so, how does it work? If not, what work would be needed to make it possible?
Would a sponsorship help to implement this?
from swift.
-experimental-hermetic-seal-at-link
requires -lto=llvm-full
or -lto=llvm-thin
.
Trying -lto=llvm-full
or -lto=llvm-thin
, with or without -experimental-hermetic-seal-at-link
, on a Hello World program (swift package init --type executable
), results in
$ swiftc -target wasm32-unknown-wasi -lto=llvm-full -lswiftCore -static Sources/main.swift
wasm-ld: error: /home/bastian/Downloads/swift-wasm-5.8.0-RELEASE/usr/share/wasi-sysroot/lib/wasm32-wasi/libc.a(__main_argc_argv.o): undefined symbol: main
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
Any pointers @kateinoigakukun @MaxDesiatov?
from swift.
Whoever is interested, the detailed plan for improvements is available in this Swift Forums thread:
Phase 1
- swift driver changes for providing flags for LTO
- swift driver changes to invoke the clang linker driver properly for enabling LTO
- swift frontend changes to emit LLVM BC instead of object files
Measurements for this phase would be interesting as they would identify the benefits of extra IPO of the IRGen (generic, non-language specific optimizations)
Phase 2
- changes to LLD to add support setting multiple compiler pipelines for LTO
- changes to swift frontend to support multiple pipelines
- changes to swift frontend to emit SIB instead of IRGen
- changes to LLD to setup a swift pipeline if a SIB is encountered, call back into swift to SILGen
- changes to the pipeline to do IPO with language specific considerations - e.g. late monomorphisation of generics, late devirt of calls
Parts of phase 1 are currently in review as swiftlang#31146.
from swift.
The progress report is now posted in this new Swift Forums thread.
from swift.
Is it possible to use SwiftWasm with -experimental-hermetic-seal-at-link
?
Also, it looks like Apple is working on "Embedded Swift": https://forums.swift.org/t/embedded-swift/67057
from swift.
As far as I know, there is no major blocker though we might need some minor adjustments specific to wasm. This is "just" a priority and human-resource problem. At this moment, upstreaming has a higher priority
from swift.
We are integrating SwiftWasm into Wasienv and realized that creating a file for a simple hello world is 42Mb (see attached file at the bottom of this comment).
You can follow the steps to compile easily Swift to Wasm easily here:
https://docs.wasmer.io/ecosystem/wasienv/compile-swift-to-wasm-wasi
How Wasienv works
This is what wasienv is doing under the hood:
wasienv install-swift
:- Installs SwiftEnv (if is not already installed):
git clone https://github.com/kylef/swiftenv.git
(1) - Installs SwiftWasm for SwitfEnv:
swiftenv install https://github.com/swiftwasm/swift/releases/download/swift-wasm-DEVELOPMENT-SNAPSHOT-2020-03-08-a/swift-wasm-DEVELOPMENT-SNAPSHOT-2020-03-08-a-osx.tar.gz
(2)
- Installs SwiftEnv (if is not already installed):
wasiswiftc example.swift -o example.wasm
:
Perhaps we are doing something wrong?
Simple Example - 42Mb
The file that wasiswiftc example.swift -o example.wasm
generated for the following is 42Mb.
if CommandLine.arguments.count < 2 {
print("Hello, WASI!\n");
} else {
print("Hello, \(CommandLine.arguments[1])\n");
}
from swift.
@syrusakbary Thanks for trying Swift for WASM!
Wasienv looks doing right. swift-wasm-DEVELOPMENT-SNAPSHOT-2020-03-08-a
toolchain is already old.
Please try the lastest toolchain from here https://github.com/swiftwasm/swift/releases
Recently I worked on reducing binary size, and succeeded to make it around 10MB.
I know it's still very fat, so I'm working on dieting more.
from swift.
Cool, let me update the release and I'll put my findings after.
Thanks for the quick response!
from swift.
wasmtime has opt flags --optimize
and -O2
btw which not good described in docs. Did you try it?
from swift.
Yup, tried them both, but the timings are exactly the same than without those flags (mainly because -O2
is already the default in wasmtime).
You can also try them locally, following instructions here: https://docs.wasmer.io/ecosystem/wasienv/compile-swift-to-wasm-wasi
But the timings are: .2s for Wasmer and 1s for Wasmtime (with opt flags turned on)
from swift.
I tried reducing a 7.9mb
wasm binary with the following results:
wasm-strip
=>32,91%
reduction (5.3mb)wasm-opt -Os
=>45.57%
reduction (4.3mb)- a combination of strip then opt =>
56.96%
reduction (3.4mb) - a zip of the above measures
1.2mb
=> a84,81%
size reduction- this is not the worst, but still not suitable for embedded devices
There's a meta point on building Swift with size optimizations. Swift LTO -lto=llvm-full
, which as @kateinoigakukun mentioned could potentially shave off another 20%, but I couldn't make it work (fails on linking). So the best I could do at the moment is compiling any swift with the -Osize
flag.
Are there any other ways to reduce the binary size? @MaxDesiatov wrote a nice summary of what improvements could be made in future. With respect to that, I saw that stdlib dropped the ICU dependency. Is this change already in effect? What's the word on WASI custom allocators?
Attaching results for reference - SizeOptimization.zip
from swift.
One blocker that I know of for implementing a minimal standard library is swiftlang#63603. There may be more of course to be found when this one is resolved.
from swift.
It indeed looks like LTO hermetic seal may be the most promising option here. I see that @kateinoigakukun actually did a talk about this option last year.
Is the work towards supporting this option for SwiftWasm blocked by something in particular or is it a case of it "just" needing more hands/eyes?
from swift.
Related Issues (20)
- SwiftWasm 5.10 release HOT 4
- Make Swift SDK host platform independent HOT 2
- Fatal error during compilation (Functions with 'no-prototype' attribute must take varargs) HOT 2
- The test executable fails to run due to call stack exhausted (swift-wasm-DEVELOPMENT-SNAPSHOT-2024-03-22-a) HOT 5
- XCTest requires access to host file system HOT 2
- XMLParser use requires shim to include libxml2.a HOT 3
- Cannot install swift-wasm-DEVELOPMENT-SNAPSHOT-2024-03-30-a-ubuntu20.04_aarch64.artifactbundle.zip HOT 3
- Roadmap: WASI Preview2 Support
- Support Swift SDK in vscode-swift
- Cannot import Foundation when using a host platform independent SDK HOT 6
- undefined symbols (RegexParser) HOT 2
- Thread and RunLoopMode.common in not available on WASM HOT 1
- SwiftWasm 6.0 release
- wasm-ld: error: swiftrt.o: undefined symbol: swift_addNewDSOImage HOT 2
- Overwriting a file with `Data.write(to:)` causes an error
- Embedded Wasm Build Is Needlessly Slow HOT 1
- Repair swift-corelibs-foundation build
- Bundle swift-testing in toolchain HOT 1
- Global Constant Strings Unavailable with Reactor Model HOT 9
- Migrate integration tests to use Swift SDK instead of assuming toolchain style distribution
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from swift.