New Open-Source G2 Editor RFC

nord_balls · Posted: Tue Jul 23, 2024 11:13 am Post subject: New Open-Source G2 Editor RFC

Hi all,

It seems that efforts to run brunov's open-source G2 editor on Apple Silicon are not moving forward. The current state is the editor is built to expect libusb binaries in /usr/local, but Apple in their infinite wisdom has made that a readonly FS so installing libs with brew are not visible to the editor build.

In theory this could be resolved with a rebuild. However, the Delphi compiler only runs on Windows, and the free Community Edition cannot cross-compile, meaning maintenance for non-Windows platforms requires an expensive license.

I've made some progress investigating a port of brunov's code. Currently my efforts are on a Java library, with the idea that said library could be used to build both desktop and mobile UIs using IDEA tech. So far I have basic communication going over USB to the G2 so I'm getting a good idea of what a library and an application would require.

I thought now would be a good time to ask the community about what would be the most maintainable approach. I have professional experience with Java and Haskell; I've ruled out Haskell for a number of reasons. Java has the advantage of being cross-platform, and there are truly free OpenJDK VMs to use so it is not dependent on Oracle.

However, TypeScript is an attractive option. I have basic familiarity and spent a little time with usb libraries there. My async/await fu is poor though, so when it comes time to stand up threads for dispatching messages it could get ugly. For standalone UIs I guess it's Electron shell or some other tech. The final advantage is being able to use pch-viewer code from ian-s and friends more or less unchanged.

Thus I'm looking for input, and then after that, help. I think G2 devotees such as myself are pretty desperate for a maintainable editor so I think moving away from Delphi is the right approach. However I can even be talked out of that if there are better ways to approach the pascal code!

The main question is what tech, and my proposal is Java, or TypeScript.

Java:
PROS:
- cross plaf, good desktop UI support, possible mobile support
- good support for threading + messaging
- usb library is fine (even there a little annoying getting apple ARM to work)
CONS:
- "eew java" -- not the most popular lang in open source. However the UI could be written in Kotlin with the library in java. The library itself could even be in Kotlin ...
- no code to reuse, everything must be ported. Not too bad in itself but more work.

TypeScript:
PROS:
- lots of devs out there, "modern"
- also cross-plaf, possible browser-based execution via webusb (???)
- usb library is good
- reuse pch-viewer code
CONS:
- Still javascript, so not sure how robust the messaging layer will be. Probably fine but I personally don't have exp with that
- Desktop UIs via Electron is ... unknown how good an experience that will be?
- Not sure what mobile path is (but there probably are many)

And finally,
Delphi:
PROS:
- done, working where it works
CONS:
- very few devs available I think
- Weird, non-modern lang (I'm more familiar now that I've ported some of the USB code, but man what a weird language. Case insensitive what what what??)
- $$$ non-open compiler toolchain for cross-plaf build
- Windows-only IDE

Cheers,
Stuart

Burp · Posted: Fri Aug 09, 2024 9:12 am Post subject:

Hi Stuart,

A realy great idea and indeed very welcome! Would love to edit my G2 with my Mac M1 and it's nice screen.

I also thought about this and i also did some testing with javascript and webusb. Had some very basic communication working but nothing ported yet.

I also had the idea of using (and improving) ian-s pch-viewer code for the UI part. At least as a starting point. I allready did some changes there to have some feel for it.

My take would be web-based (so javascript/typescript) so there are lots of devs and that would make it easier to maintain. But the USB part should be tested first, if it could work.
There are options to make webbased apps into 'native apps' like the electron approach.

Allthough i'm a bit hesitated to jump into a time-consuming project, maybe we could join forces? And hope some more devs will jumps in?

Cheers!
-Jan (NL)

By the way, i am a (frontend) developer for a living.

nord_balls · Posted: Fri Aug 09, 2024 5:46 pm Post subject:

Hi Jan,

blue hell · Posted: Fri Aug 09, 2024 6:41 pm Post subject:

I'll not contribute directly as I've moved away from the nords - but still two g2's are in my rack and I will be happy to crittically try things out on an old 32 bit mac (mojave 10.14.6) and a win10 pc.

Anyway - great to see something happening here!

Burp · Posted: Thu Aug 15, 2024 7:07 am Post subject:

I thought about it a bit more.

As i first thought the webUSB (and JavaScript/TypeScript) way was the way to go due to more developers can help/understand it. But webUSB is not a standard. There is a reason that not all browsers (Firefox) support it. It has some major security risks that probably will decrease the change of being a well settled standard. Although those risks in our case are small, the fact that it is not a well settled standard is a problem for our goal.

So maybe the Java route is the better. I'm not familiar with Java but maybe you can reflect on this? Is it a safe route to go for support, opensource etc? My guess is, it is. And is it possible to compile/run Java applications on other devices then desktop (macOS,Windows,Linux)? For example android and iOS tablets, or simple raspberry Pi's (which run a Linux clone)?

How is Java at GUI's?

Apart from the technical thoughts, i still have hesitations to join full in. Because lot of other responsibilities and hobby's (music is one of them Wink

.

But what i definitely like to do is, same as Blue Hell, test it with my stack (Expanded G2 keys, MacBookPro 14" M1). And i'd also can read/review your code now and then to see if i understand it. And maybe, maybe... i'd help out with the UI part. That's where my heart is more than the lower level tech stuff.

What i also can do is fork ian-s patch viewer and commit my changes to it.

A a first step: can you add some minimum information to your repo how to run it and test it? Then i'm glad to test what you have build.

nord_balls · Posted: Fri Aug 16, 2024 4:24 pm Post subject:

NordG3 · Joined: Apr 29, 2021 Posts: 6 Location: On the internet

Burp · Posted: Thu Aug 22, 2024 10:57 pm Post subject:

Burp · Posted: Thu Aug 22, 2024 11:00 pm Post subject:

NordG3 · Joined: Apr 29, 2021 Posts: 6 Location: On the internet

Burp · Posted: Thu Aug 22, 2024 11:35 pm Post subject:

Burp · Posted: Fri Aug 30, 2024 11:23 pm Post subject:

I managed to get ian-s pch-viewer (with some changes) into an electron app. I tried contacting him (pm) before i put this on a GitHub repo, i don't won't too pass him. But no reaction yet.

This relative simple experience with electron was good. It feels like a good way too proceed. I also tried their basic webUSB example too see if my Nord is seen by it (nothing more). And that works.

Anyone has an idea to contact ian-s?

blue hell · Posted: Sat Aug 31, 2024 12:17 pm Post subject:

Burp · Posted: Mon Sep 02, 2024 6:09 am Post subject:

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

Pasted from the other thread: I'm starting to nibble around the edges of getting comms working with the G2, possibly as a basis for an editor. Partly to get me further into XCode.

Looking at the Delphi open-source editor, it seems to use libusb. Not sure that's required though.

Possibly the iousb stuff Apple has might be a better mechanism?

Anyone have experience with this stuff? Any pointers?

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

For what it's worth, I've got something based on the example code linked below working. It's seeing the G2 on USB. So - first step taken!

https://github.com/opensource-apple/IOUSBFamily/tree/master/Examples/USBSimple%20Example

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

Currently wondering whether to jump for something based on JUCE, Objective-C or Swift.

Maybe if there’s a good G1 editor with the right framework, that’s maybe the way forward?

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

Slow progress, but does seem to be progress.

I'm currently attempting to get the initialisation message out of the Mac, into the G2...

I've got WireShark running on my old Mac with the original G2 editor. That combined with info on the link below - it's tying up.

https://www.bverhue.nl/G2EditorHelp/G2_messages.html

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

Update: I have some 'C' working on the Mac, which is reading/writing data to/from my G2!

NordG3 · Joined: Apr 29, 2021 Posts: 6 Location: On the internet

Can someone tell me if the Windows G2 Editor v1.62 available on the NordKeyboards.com homepage is known functional on new Windows 11 x64 Intel computers? If not I would love to have a Windows Delphi G2 FMX app recompiled for my G2 Engine I just bought. I kept getting crashes on the G2 FMX app on macOS Sonoma 14.6.1 that required restarts after the editor disconnected me but I enjoyed the G2 for awhile.

Burp · Posted: Mon Oct 07, 2024 10:48 am Post subject:

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

By the way, I’ve ditched Libusb and am now going direct to IOKit.

For those who reckon the Delphi editor is suffering from the Libusb thing, I could potentially share my IOKit work in ‘C’, which could be used to modify the Delphi editor, removing Libusb?

I don’t have a Delphi compiler though.

Starting to attempt to learn Swift for the GUI, but it’s a slow process. Someone with more experience who was willing to pick up the GUI bit would push things forward significantly I reckon.

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

I'm hoping to publish my code in GitHub soon as open-source.

It's very much a starting-point. Most of the problems I've had, have been related to IOKit. I feel like I have a pretty good solution to that now.

It's all 'C', which might need to be translated to Swift or Objective-C for GUI. Lots of message types to add. Struggling with the actual patch data message, since bverhue's work is great and has been immensely helpful, but doesn't describe the patch data responses format/protocol.

Another problem I've hit, is when loading a new patch, the G2 refuses to give me patch data for the new patch unless I power cycle it. There must be a special message I need to send to reset it. Init(ialisation) doesn't clear that problem. I'll probably need to fire-up Wireshark again to see what happens on the original editor. ...or I see how bverhue deals with it in Delphi, if that app does in fact deal with it.

ChrisPurusha · Joined: Sep 09, 2024 Posts: 17 Location: UK

Here's the work in progress. This pretty-much goes through the initialisation sequence and dumps the data to the console as-is. Then polls for continuous data such as volume or LED status.

https://github.com/chrispurusha/G2-Edit

If anyone wants to help with the GUI side of things, please do drop me a line.