Why

I ran into a somewhat interesting problem at work recently.

Our flagship application had to ingest a new fixed-width file format. In itself this should be a pretty simple task, but our existing infrastructure utilizes a mess of nested SQL stored procedures, and I did not want to contribute to that any longer. I had a talk with my manager, and he agreed to give me the latitude to explore other alternatives.

To me, anything felt like an improvement. Debugging nested SQL stored procedures are a pain in the ass, and trying to figure out which line caused a problem with your "parser" is even more painful.

Since we're a financial institute, I figured that speed, consistency and safety were non-negotiable. Yep, you know it already, I'm writing it in Rust.

But this begs the question, how do I plan on calling whatever I write in Rust, from C++? In our existing codebase, we have a bunch of methods we use to do that. Anything from FFI's to actually just launching the exe directly. I did not want to launch an exe as there's really no UI/UX component to it. I didn't want to just have it as a CLI either, as that felt a little messy. I finally settled on the FFI approach.

How

I won't show actual production code (obviously), but just imagine that I have a function that accepts a &str, goes through a bunch of nom parsers, and spits out a ParsedData struct. ParsedData will get serialized into json, utilizing serde_json. Let's call that parse_string.

Setting up `cbindgen`

The cbindgen documentation is pretty much all I referenced. Here's the speedrun version:

Install cbindgen using cargo install cbindgen
Create build.rs in root directory of the project
Copied the example script in the documentation
Added cbindgen to build-dependencies (make sure to check for the latest version!)
Expose a function using #[no_mangle] pub extern fn
Changing my package to compile into a staticlib, as I needed .lib (working in Windows environment)

Exposing 😲 Rust to C++

I only wrote one "bridge" function. It takes in a path to a file, and returns a String. On the C++ side, it passes a CString to the function, and receives a huge CString back. On the Rust side, it takes in a c_char and similarly returns a c_char. This function then checks to see if the file exists. If it does, it will read the file as a string and pass it onto the parser and expect a ParsedData struct back.

In order for the function to be picked up by cbindgen and have its equivalent function signature generated, you simply have to declare it with #[no_mangle] pub extern fn. You will notice that this generates a function prototype in the header file that you previously defined in build.rs.

Some oddities

Not playing nice with Rust > 1.69.0 - this, this, and this. I'm no expert, but I think the gist is that instead of using dynamic linking, it's now doing static. However, I'm unable to include the static library required (ntdll.lib), because reasons? I think if I got the wdk or something, I could then do that, but that's too much of a lift for my development team honestly. So I decided to just use 1.69.0 for now while I figure that out.
Target - I ended up having to compile for i686-pc-windows-msvc, as it had to work with MSVC ABI. Furthermore, the application only ran in 32-bit and so I had to target 32-bit as well.

Adding it to existing project

This was probably the easiest part, but took me the longest. The steps were basically:

Copy the generated .lib file into an existing folder containing all your libraries (if you have one), or simply add the folder containing the .lib file to your Library Paths.
Include the library in Linker > Input > Additional Dependencies
Add generated .h file
Call function as you normally would
??
Profit

Automatic builds?

We use Azure.

We like pipelines & automated builds.

Nathan write pipeline.

Microsoft rust-pilled, windows-latest contains rustup.

Nathan only had to install older version and set target.

Nathan happy because easy pipeline.

Wow!

The results were great. I got to parse that file insanely quick because of Rust, but I could also use it in our existing C++ application. I'm really excited at the prospect of slowly shifting more important and intensive duties to Rust, rather than keeping them the way they are written now. Frankly, I'm having some crazy ideas to rewrite our business logic in Rust and simply expose them to other languages with cbindgen, pyo3, cxx, etc. Have one safe & consistent library to do all the important heavy lifting, and use other languages to do the less important and things that change more (UI/UX).

I'm still not done though. Next I've got to use that json string in C++, and I'll have to hook that into our existing stored procedures. Stay tuned to see if I figured out another way around that!

Table of Contents

Why