Swift Ruined My Weekend

Swift Ruined My Weekend

So as you may know if you follow me on the fediverse (@[email protected] if you aren’t), I like Swift. This isn’t just because I happen to like Apple products, it’s actually a dank as hell language and has (so far) kept me from falling into the Rust cult abyss. The fact that it is plug-and-play integrative with C is a major plus that just makes me endlessly happy due to my low level shenanigans.

Shenanigans aside, I have recently discovered something that blows my mind on Swift. And not in a purely good way. Swift does not just pass stuff through to C without effort. There is some fuckery in the backrooms, and it can catch you out if you’re not watching.

A Tale of FFTs

I work on some pretty cool anomaly detection kit for Hyprfire, which makes my job less of a job and more of a convenient way to get paid for the random code I do on computers usually. More recently, I’ve been looking for new and more entertaining ways to detect weird things. Currently on my plate is a project involving Fast Fourier Transforms (see this sick video by Veritasium if you’re not sure what they are). FFT’s are hard, we’re talking complex number, divide-and-conquer algorithm stuff.

I’ve cheated a bit and pulled in a library called KISSFFT by Mark Borgerding, which is really neat, simple, licensed under the BSD 3-clause (a big plus in my book), and written entirely in C. It’s not the most blindingly fast FFT implementation, but it takes less time to FFT a data sample than it does to MD5 it, so as far as you are concerned it’s probably fast enough.

Anyway, so I’ve pulled this library in, and wrapped it in a Swift class. All this class does is group the FFT config construct with the input and output data arrays, implemented here as UnsafeMutablePointer<kiss_fft_state> and UnsafeMutableBufferPointer<kiss_fft_cpx> respectively (I was not kidding about the complex numbers). These, when the FFT proc is invoked, will be passed to the kiss_fft function as UnsafeMutablePointers, in the Buffer pointer case by a typecast. This is necessary as the actual fft function is implemented as:

void kiss_fft(kiss_fft_cfg* cfg, const kiss_fft_cpx* fin, kiss_fft_cpx* fout);

The important thing here is that Buffer pointers and normal pointers are different in Swift, although they are syntactically indistinguishable in C. Keep this in mind, because it comes back like a backhanded slap later.

This is all normal so far. I implement a test based on data from a purely C example program that does FFT on some stock data. The idea is that I expect the Swift version to produce something that is mostly the same, give or take 5% on the individual values I get out.

That done, I try running the test, and I get the following error:

exited with error 10

For those that don’t know what error 10 is, that’s your old pal SIGABT. We got an abort, the program is telling us “Oh man I aint right, y’all better kill me”, meaning we’re wildly outside memory beyond the normal conditions of a SIGSEGV. But my test is the same as the example program. Same data and everything.

What in the wide world of sports is going on?

Somebody call BHP Billiton, it’s Time to Dig.

So after trying to beat the generic command line Swift compiler into telling me what is up, I opened my project in XCode to see if that helps.

Fun fact, if you include C in your Swift package as a target, the lldb debugger will just happily include it in the debug. Neat! I was expecting something far more obtuse than this, but we’ve identified the line in question.

Bad news is it’s in the C library. Looks like there’s a de-reference that is pointing way outside memory for some reason.

Interestingly, the program manages to do some loops of this work recursion program before it dies. So it’s not completely wrong. And sure enough, jumping back a few ticks on the stack, we see that both Fout and f are set to valid complex structs:

Okay… maybe the struct array has been truncated somewhere? Maybe I’m not allocating it enough elements? I know it needs 2048 complex structs in both the in and out arrays to work. So let’s pop back to the Swift to have a look. Up some more steps in the stack and we see the following:

No… that’s 2048 elements…

And then, I saw it.

Look closely. f is pointing to something in 0x16...., while startingData is pointing to 0x10..... That’s not even close to the same memory address. What the heck?

Sure enough, as soon as we hit C land, the pointer has mysteriously moved a great distance. How? What? Why? Whomst?!

Swift, of course. But why? Why backstab me like this?

Look closleyer. fin is a const kiss_fft_cpx*… and is the only pointer with a const modifier. Hmmmmm…

My first thought was: “Hey, maybe Swift considers this a UnsafePointer, not an UnsafeMutablePointer?“. So I tried changing it up with an UnsafeBufferPointer, and then this happened:

Curiously, this didn’t work at all. In fact Swift was furious. The reason was that Swift considered UnsafePointers immutable in content as well as location. And that’s when it hit me: Swift is treating the const pointers as immutable. So I changed the definition of kiss_fft to:

void kiss_fft(kiss_fft_cfg* cfg, kiss_fft_cpx* fin, kiss_fft_cpx* fout);

I also returned my Buffer pointers to their previous, mutable state.

And hey presto, it works!

Et tu, Swift?

So why has Swift stabbed me in the back like this? I thought this was clear, I’m passing an input array as a pointer as per C standard, and it’s const so that I don’t change where the start of the input array is (which is important for many things, not just FFTs). I’ve followed the rules, what the heckers?

The answer is that Swift is trying to be helpful and protect our nice, sane Swift memory space from the uncaged madness in C’s world. Swift is percieving the const keyword as an indication that the pointer is immutable, but in classic Swift style it is assuming it is immutable in content as well as location. To protect against accidental rewriting of content, Swift has copied the contents of the pointer destination to a safe zone somewhere else on the heap, such that any change to it will be ignored back in Swift-space. This safe zone does not contain the contiguous array of complex structs, just the 0th one. As such, the kiss_fft algorithm does about 6 steps of fandango on core before leaving addressable memory altogether and getting shot by the kernel. Getting rid of the const stops Swift from trying to help us like this, allows the C to walk to the other complex structs, and resolves the problem.

So there you go, Swift can try to be helpful with your memory management in Swift->C cross-overs. This “being helpful” can trip up the average C programmer who is used to direct and total control of the memory space, and result in bizarre, inexplicable memory issues. In other news, if you are using C in your Swift environment, I would consider ingesting it into your Swift package constructs rather than simply building them elsewhere and linking them as system libraries so that the XCode environment can strap your whole codebase to the llvm debugger it has running, makes your life way better.

TL;DR:

Swift tries to “help” with memory management in C/Swift cross overs. Watch out, it’ll pimp slap you when you least expect it.