Nintendo Switch Basic Support #1108

racerxdl · 2020-05-13T04:14:28Z

I added basic Nintendo Switch support to TinyGo.

It still relies on devKitPro and libnx to interact with Horizon OS (The Nintendo Switch Operating System), I hope that changes soon.

Still, that does work. At first I added the libnx calls here, but I found out its easier to make an external library to that. So I created this:

https://github.com/racerxdl/gonx

Without gonx the application can run inside nintendo switch, but can't use the display or anything until we implement full golang calls for the Switch IPCs.

I also created this: https://github.com/racerxdl/go-switch-examples

I will port each example from the switchbrew examples repo ( https://github.com/switchbrew/switch-examples/ ) and implement the nescessary calls on gonx as needed.

For compiling to nintendo switch its simple by using:

tinygo build -target nintendo-switch -o hello-world.elf

For running on real devices a NRO file needs to be generated, that can be done with elf2nro (which is inside devKitPro) or with linkle:

linkle nro hello-world.elf hello-world.nro

Here is a video of working: https://twitter.com/lucasteske/status/1259650256897298432

And the vt52-example inside examples folder prints this:

trevor403 · 2020-05-13T06:21:57Z

Wow!

aykevl

I did a first pass over the PR.

There is a bit of code in builder/build.go that I'm not so happy about, I wonder how that can best be fixed? Ideally no environment variable is needed (apart from making sure the compiler is included in $PATH), but otherwise the best place to configure this would be compileopts, not builder.

.gitignore

builder/tools.go

src/machine/machine_generic.go

src/runtime/print_nintendoswitch.go

src/os/file_nintendoswitch.go

racerxdl · 2020-05-13T15:19:01Z

I did a first pass over the PR.

There is a bit of code in builder/build.go that I'm not so happy about, I wonder how that can best be fixed? Ideally no environment variable is needed (apart from making sure the compiler is included in $PATH), but otherwise the best place to configure this would be compileopts, not builder.

The issue is that there still need to have the devkitpro env to get the libnx include path. It's always inside devkitpro folder, but you dont know where is installed. All devKitPro projects expect a DEVKITPRO environment to build.

aykevl · 2020-05-13T21:17:57Z

The issue is that there still need to have the devkitpro env to get the libnx include path. It's always inside devkitpro folder, but you dont know where is installed. All devKitPro projects expect a DEVKITPRO environment to build.

I was playing around with it and you're right, even the bundled compilers haven't been configured to include libnx by default. Which makes things a whole lot more complicated.

I tried running examples/nintendoswitch in Yuzu but it doesn't work for me. That may also because of my Yuzu build (which I only barely managed to get to work). Does the example work in Yuzu?

racerxdl · 2020-05-13T21:31:28Z

The example does work on Yuzu (the screenshot is from yuzu). But I don't remember if you need the basic Nintendo Switch dump to make it work.

To avoid legal issues with Nintendo proprietary assets, to run commercial games on yuzu you need to dump keys from your console and some content from the NAND flash. That's a requirement for commercial games because they're signed and encrypted. The NRO files are neither, so I don't know what's the minimum requirements to run. I will check here (since I have everything) what's the minimum setup to get it working.

aykevl · 2020-05-13T21:48:14Z

Can you maybe share a working .nro file that I can test? To determine which is the cause (the build or Yuzu).

racerxdl · 2020-05-13T21:51:28Z

Sure, this one works here:

vt52-demo.nro.zip

aykevl · 2020-05-13T22:14:20Z

Well that version sadly doesn't run, so I'm afraid my Yuzu build still doesn't work.
One time it even crashed:

Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
dynarmic: POSIX SigHandler: Exception was not in registered code blocks (rip 0x007f596a4da8bb)
fish: “yuzu tmp/vt52-demo.nro” terminated by signal SIGSEGV (Address boundary error)

(fish is my shell)

racerxdl · 2020-05-13T22:16:40Z

Hmm thats odd. As I remember it gives a message box saying what you need to run, not crashing. Maybe its some issue with wayland? Here I'm using Xorg instead, I never tested on wayland.

aykevl · 2020-05-13T22:20:22Z

This is what it says on startup, is that a problem?

racerxdl · 2020-05-13T22:21:46Z

That's the thing, for commercial games I'm 100% sure you need everything. Homebrew need only a subset. In about an hour I can confirm what's needed.

aykevl · 2020-05-14T00:18:26Z

I have also managed to produce a test.nro that works in yuzu, so I have all the tools I need for local testing.

aykevl · 2020-05-14T00:38:01Z

Played around a bit with avoiding libnx, but all I got was a panic in linkle:

thread 'main' panicked at 'Invalid module offset 3531603969', <::std::macros::panic macros>:5:6
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

src/syscall/syscall_libc.go

deadprogram · 2020-05-22T11:21:11Z

@racerxdl can you please rebase this branch against dev to resolve the merge conflicts? Thank you.

racerxdl · 2020-05-22T19:47:11Z

Done @deadprogram ! I also squashed all commits.

danielhe4rt

i never doubt about teske work so its approved

enjoy running golang on switch

aykevl · 2020-05-28T00:24:29Z

We recently changed compiler/compiler.go in #941 so I'm afraid this will need another conflict resolution.

builder/build.go

src/syscall/syscall.go

racerxdl · 2020-05-31T19:11:00Z

@aykevl just changed the way the PIC stuff reaches the compiler. Now it goes on TargetSpec and comes from json spec. I modified the original NewTargetMachine to read the RelocationModel from TargetSpec and select the appropriate one (or static if invalid defined)

deadprogram · 2020-06-20T10:07:42Z

Hello, everyone. I would love to help get this PR merged before the next release. Seems like there are again (or still?) some merge conflicts to resolve.

After that, what else will be needed to get it into a mergeable state? Thank you.

racerxdl · 2020-06-21T00:35:23Z

Done @deadprogram. Not sure what's next.

It currently works by using Dev Kit Pro for Nintendo Switch in a env DEVKITPRO. See https://switchbrew.org/wiki/Setting_up_Development_Environment Basic Nintendo Switch to do a Hello World * Added Integration with libnx to be able to print a hello world Add printhello example Cleanup. using devkitpro as default for now Linking libnx with lld proved to be hard. Then for now we will default to devKitPro and print a message if we cannot find a DEVKITPRO environment pointing to DKP root Change default gc to extalloc move calls to gonx and redirect default panic Default printstring now outputs to SvcOutputDebugString which prints on Emulator Console Remove libnx from submodules Changed hello world by vt52-example Add buffered output to putchar

aykevl

If possible, please add a smoke test for this PR. It will make it obvious when there is a compile-time breaking change.

aykevl · 2020-06-23T20:33:37Z

builder/build.go

@@ -27,11 +28,16 @@ import (
 // The error value may be of type *MultiError. Callers will likely want to check
 // for this case and print such errors individually.
 func Build(pkgName, outpath string, config *compileopts.Config, action func(string) error) error {
+	var err error
+	var machine llvm.TargetMachine


It looks like you didn't undo these changes, which are now just noise.

Is there something needed still for this comment?

aykevl · 2020-06-23T20:36:35Z

builder/build.go

+		dkp := os.Getenv("DEVKITPRO")
+		// If Nintendo Switch, add libnx
+		if config.Target.Linker == "devKitPro" {


I still don't really agree with adding code here that is specific for a particular board. I was under the impression that the special linker and libnx wasn't necessary for initial Switch support?

It is needed. The support is complete useless without being able to link libnx. I tried libtransistor (which is made using LLVM) but its basically abandoned and has some issues.

aykevl · 2020-06-23T20:38:08Z

compileopts/target.go

+		GDB:             "gdb",
+		PortReset:       "false",
+		FlashMethod:     "native",
+		RelocationModel: "static",


This is unnecessary: it defaults to static already.

Is action required on this comment?

aykevl · 2020-06-23T20:39:23Z

compiler/compiler.go

@@ -109,7 +110,18 @@ func NewTargetMachine(config *compileopts.Config) (llvm.TargetMachine, error) {
 		codeModel = llvm.CodeModelLarge
 	}

-	machine := target.CreateTargetMachine(config.Triple(), config.CPU(), features, llvm.CodeGenLevelDefault, llvm.RelocStatic, codeModel)
+	switch config.Target.RelocationModel {


Suggested change

switch config.Target.RelocationModel {

switch config.RelocationModel() {

aykevl · 2020-06-23T20:39:50Z

compiler/compiler.go

+	case "dynamicnopic":
+		relocationModel = llvm.RelocDynamicNoPic
+	default:
+		relocationModel = llvm.RelocStatic


This should return an error instead of silently defaulting to static with an unknown relocation model.

Looks like something still required here?

aykevl · 2020-06-23T21:04:45Z

src/runtime/runtime_nintendoswitch_heap.go

+const heapSize = 16 * 1024 * 1024
+
+//go:extern _stack_top
+var stackTopSymbol [0]byte


Where is this defined?

That's from devKitPro libc. I will put a comment there.

aykevl · 2020-06-23T21:06:01Z

src/runtime/runtime_nintendoswitch_svc.go

@@ -0,0 +1,17 @@
+// +build nintendoswitch


It seems to me that this could be in the runtime_nintendoswitch.go file, or is there a reason to keep it separate?

aykevl · 2020-06-23T21:07:07Z

src/syscall/syscall_baremetal.go

@@ -12,6 +12,7 @@ package syscall

 // A Signal is a number describing a process signal.
 // It implements the os.Signal interface.
+


This looks unintentional?

aykevl · 2020-06-23T21:09:36Z

src/syscall/syscall_libc.go

 func Close(fd int) (err error) {
-	return ENOSYS // TODO
+	errN := libc_close(int32(fd))


What are all these syscall changes for? Are they necessary for initial support?
It seems better to me to do it in a separate PR and do it properly (without all the TODOs here).

Yes, for nintendo switch that is actually needed (it does have file I/O). Actually I'm not sure why they weren't implemented like that since the picolibc included on tinygo has that libc calls as stubs.

aykevl · 2020-06-23T21:10:24Z

targets/nintendo-switch.json

+  "goos": "linux",
+  "goarch": "arm64",
+  "compiler": "aarch64-none-elf-gcc",
+  "linker": "devKitPro",


Wasn't it possible to link using lld?

Also, if you can use aarch64-none-elf-gcc, would clang (with the right options) work? The less dependencies, the better.

It wasn't. LLD can't link stuff pre-compiled with GCC with custom linker script :( - It gave several errors. Also it does need DevKitPro because of the libc required for all nintendo switch deps (the DevKitPro is a modified picolibc).

We might eventually want to port devKitPro picolibc to tinygo and replace the current one, because the picolibc from devKitPro is more generic than the one we have on tinygo today. I checked if it was easy to port it, but there are some things that require changes on other tinygo targets, so I think its better a separated PR for that eventually.

Okay, now I understand the situation better.

The linker field here really refers to an actual command. I think instead of repurposing the linker field, you should do something like this:

"libc": "libnx",

That is actually the field intended for changing the libc, which is the main reason all these custom build steps are needed. You can then check for libnx (similar to picolibc) and if it is used, add the correct paths by querying the DEVKITPRO environment variable. Not how I would hope it would work, but I think that is the cleanest way to do it.

On second thought, you need to run a command in the $DEVKITPRO directory. Which is not known statically. This requires some logic to determine the correct path.

This is somewhat of a mismatch between how I intended TinyGo to work and how DevKitPro works. I'm not sure how to fix that exactly. Maybe as it is right now.

Maybe as it is right now.

@aykevl do you mean as this PR has already implemented?

Yes. I don't like it but I don't see a better way, so maybe just do it as the PR is now. It's unfortunate that devkitPro assumes the use of global variables, which isn't very compatible with TinyGo right now.

deadprogram · 2020-07-08T20:34:35Z

If possible, please add a smoke test for this PR. It will make it obvious when there is a compile-time breaking change.

Yes, please, @racerxdl is this possible? Ideally something with no external dependencies. Does not have to do much, just shows that the core compiler itself is working.

racerxdl · 2020-07-08T20:45:16Z

Hi @deadprogram / @aykevl - sorry I didn't had much time to look these days. This weekend I will try to get a look in all comments and do the nescessary fixes / comments.

About the smoke test, yes I can put a very small program just to test that everything is compiling. The thing is that the only change for Nintendo Switch is basically the devKitpro call for libnx linking, otherwise is just a normal arm64. So If I remove the devKitPro dependency for a smoke test, it will just test the ARM64 compiler part. Is that what's intended?

The last time I was working on that, I tried to make the libnx linkage externally to the tinygo (by doing this https://github.com/racerxdl/gonx/blob/dev/base.go#L6-L7 ) but that does seen to trigger a bug in CGO ( #1148 ) which @aykevl wasn't able to simulate. (I think because he is importing from a local folder and not a external library).

If I manage to do a external linkage, I can put a very small CRT for nintendo switch inside tinygo so it works as a application inside the nintendo switch. Also I think in that case, the lld can link it (just not link when using libnx).

deadprogram · 2020-07-08T20:48:44Z

Thanks @racerxdl that sounds great!

racerxdl · 2020-07-10T05:00:54Z

Hi @deadprogram / @aykevl while I work on the external integration of libnx for this branch, I did another setup that does not require devKitPro or anything, but can only print to Nintendo Switch emulator debug console.

#1226

But this will serve as a base for external linkage of libnx. I will change this PR (#1108) to draft and then when its ready again I switch to a PR again.

aykevl reviewed May 13, 2020

View reviewed changes

racerxdl force-pushed the nintendoswitch branch from f098b29 to 945c5f4 Compare May 14, 2020 02:32

cornelk reviewed May 14, 2020

View reviewed changes

src/syscall/syscall_libc.go Outdated Show resolved Hide resolved

racerxdl mentioned this pull request May 20, 2020

cgo: Add LDFlags support and float types #1125

Merged

racerxdl force-pushed the nintendoswitch branch from da74ea2 to 80385cc Compare May 21, 2020 00:05

racerxdl changed the base branch from master to dev May 21, 2020 00:06

racerxdl force-pushed the nintendoswitch branch 3 times, most recently from 551c9c6 to 6245cd4 Compare May 22, 2020 19:28

danielhe4rt approved these changes May 28, 2020

View reviewed changes

aykevl reviewed May 28, 2020

View reviewed changes

builder/build.go Outdated Show resolved Hide resolved

aykevl reviewed May 28, 2020

View reviewed changes

src/syscall/syscall.go Outdated Show resolved Hide resolved

racerxdl force-pushed the nintendoswitch branch 2 times, most recently from 76f294a to b97ea9a Compare May 31, 2020 19:03

racerxdl force-pushed the nintendoswitch branch from b97ea9a to 5c1b6d4 Compare June 21, 2020 00:45

aykevl reviewed Jun 23, 2020

View reviewed changes

racerxdl mentioned this pull request Jul 10, 2020

Nintendo Switch Minimal Support #1226

Merged

	switch config.Target.RelocationModel {
	switch config.RelocationModel() {

		@@ -12,6 +12,7 @@ package syscall

		// A Signal is a number describing a process signal.
		// It implements the os.Signal interface.

Nintendo Switch Basic Support #1108

Are you sure you want to change the base?

Nintendo Switch Basic Support #1108

Conversation

racerxdl commented May 13, 2020

trevor403 commented May 13, 2020

aykevl left a comment

Choose a reason for hiding this comment

racerxdl commented May 13, 2020

aykevl commented May 13, 2020

racerxdl commented May 13, 2020

aykevl commented May 13, 2020

racerxdl commented May 13, 2020

aykevl commented May 13, 2020

racerxdl commented May 13, 2020

aykevl commented May 13, 2020

racerxdl commented May 13, 2020

aykevl commented May 14, 2020

aykevl commented May 14, 2020

deadprogram commented May 22, 2020

racerxdl commented May 22, 2020

danielhe4rt left a comment

Choose a reason for hiding this comment

aykevl commented May 28, 2020

racerxdl commented May 31, 2020

deadprogram commented Jun 20, 2020

racerxdl commented Jun 21, 2020 • edited Loading

aykevl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deadprogram commented Jul 8, 2020

racerxdl commented Jul 8, 2020

deadprogram commented Jul 8, 2020

racerxdl commented Jul 10, 2020

racerxdl commented Jun 21, 2020 •

edited

Loading