Observations on controllers
-
So, one of the first things I've been working on after getting my hands on fuze, aside from just trying to make sense of the built-in functions, and the way the language works, was a test program for controller input.
Although there's an example one, I decided to make one from scratch, to learn a bit about how it all works.Along the way I discovered a bunch of weird, quirky things about this. Some of them might be bugs, many of them probably are not, but they do have some implications either way.
So first thing I'd like to do is show this screen capture. It's the output of my test program.
You'll note a haphazard bunch of things about it. First a bunch of text, some coloured squares, and a bunch of lines.
so let's start with the text. Look closely and you'll see the very first line is a number.
Because I wasn't sure what the full controller output looked like, and the mismatch between the existing test program and the documentation led me to believe there are some undocumented functions, this led me to see if I could somehow print the entire set of data returned by the controller input function.Sure enough, I could. But this led to the first problem. String handling. And that's where this number comes in. It's the length of the complete string returned by the input function for controller 0. (meaning the full data for ALL supported controllers would be 4 times this length).
You'll note (if that's readable in the screen capture) that it says 514 - so that's 514 characters. It fluctuates a little over time, but it's in that range.So, imagine my surprise to learn that this entire string tried to print on a single line. If you've ever used 'print' on an old 8 bit computer, or using the console output of C/C++ or even tried to print text through the windows SDK, you'll note that 'print' generally does word wrapping, and the GUI based text routines almost always include a function to draw word-wrapped text in some kind of limited onscreen region. (defined by a bounding rectangle through various means)
OK, so the standard print function doesn't do any of this. Fine, so I looked for other options. Maybe an alternate function that does it. No such luck.
I could change the text size to get it all to fit on one line, but at that scale the text is unreadable.
Since this is just some quick debug style output I was trying to find the easiest method possible.
Turns out it's a lot harder than you'd expect.OK, sure. That's a nuisance, but fine, what about string handling then? Since I'm printing the output of an entire data structure, which functions as a single long string, doing this manually requires splitting that string somehow.
So I copy the data to a string variable...
And hit my next roadblock. Either this kind of functionality is missing or it's not documented very well (or at all), but there doesn't seem to be any easy way to extract a sub-string from a longer string short of writing your own function that iterates through each character individually using array logic. (which is what I did in the end).It's certainly not an insurmountable obstacle, but it's far from convenient. Regardless of whether 'print' or some equivalent does word wrapping, the fact that it appears to be surprisingly difficult to split or concatenate strings is a bit weird. There's functionality to search for a substring within another string, but not to split or combine strings in a more direct sense?
This seems like a pretty major oversight for string handling routines, if it truly doesn't exist. (if it does, and I'm just not finding it, then the documentation might need some improvements.Which brings me to my second point; Undocumented features. You'll notice the complete set of input data includes 'velocity', 'orientation'. and 'colour.'
The inclusion of colour is fun, but, none of these things are documented at all.
Sure velocity and orientation are in the included example program, but otherwise not mentioned.So, that's clearly an oversight of some kind.
Next, we have some missing buttons for the controllers. Presumably the home and capture buttons are off limits to Switch programs in general, so that's understandable.
Plus and Minus buttons are understandable too, because FUZE uses those to handle it's own UI stuff (though minus doesn't seem to do anything while a program is running, so maybe this doesn't need to be disabled? Not sure)
The main oversight I find a little more annoying is the joycon SL and SR buttons - these are obviously not present on a Pro controller (or equivalent), but they ARE on the joycons, and they have a lot of utility if you're using individual joycons sideways (eg 1 joycon per player) or, using joycons one-handed, where they are accessible. That's 4 unmapped buttons with no clear reason for it. (each joycon has an SL and SR button)It's not the end of the world, but it would be nice to be able to use as much of the controllers as reasonably possible.
(As an aside I'd love to mess around with that depth camera or whatever it actually is - without official switch dev documentation it's hard to be sure, but I can understand why that's not implemented. )
So, that seems like a few things already, but... There's more.
And here's where we get into things that look like they may genuinely be bugs, not just something akin to 'feature requests'So far I've been using FUZE without a USB keyboard, so everything is done with controller or touchscreen input. This works well enough (though if I have one complaint it's that there's no dedicated button for 'delete' only 'backspace'. It's not an important one, but it does end up being a minor nuisance fairly often.)
There is however some odd behaviour with single joycons.
If you go to the main menu and into the switch's controller settings, then change the order such that player one is set to use a single joycon (as opposed to a pair), basically the entire FUZE UI becomes unusable. This isn't a major issue because FUZE quickly tries to pair the other joycon, but... It's still behaviour that probably isn't ideal, for something many users do attempt every now and then.Oh, also about orientation and velocity - experiments with these values tell me that orientation is a normalised vector. Now I don't know the actual hardware in a switch controller, but I would have to presume it's at least as capable as that in a Wii remote with Motionplus add-on.
This means at a minimum the raw hardware likely consists of a 3 axis accelerometer, and a 3 axis gyroscope.But what is fuze actually reporting? Are these direct readings from the sensors, or is there some kind of pre-processing involved, and if so, to what extent? (again, the kind of thing that would be good to mention in the documentation)
What I noticed is that the 'velocity', is a 3 dimensional vector, which, if you calculate it's length, will generally read to be about 9.8 give or take some degree of error while the controller is stationary.
This would appear to be gravity, which is what you'd expect for accelerometer output. The specific value would suggest this output is calibrated to newtons or m/s
Now, that's fine, nothing wrong with that at all, except, if this is what I think it is...
This value should be named 'acceleration' not 'velocity'. Because it's reading gravity as a fairly constant value, where if it was actually a velocity reading, the value would be constantly increasing. (gravity is acceleration, not velocity). Clearly this appears to be misnamed.Orientation, when treated the same way appears to be a normalised vector. This has one major downside that I can see - it only gives you 2 axes of rotation; You cannot determine the roll of the controller around it's long axis this way. Not in and of itself a disaster, but it means that at best you can only ever hope to do 5 axis tracking with the data provided, not full 6 axis tracking.
Not sure if this was your decision when creating this or whether the hardware really reports it this way, but it's an implied limitation that's worth thinking about.While I'm at it, another 'nice to have' feature I'd like to mention is some way to determine if the attached controller is a joycon/joycon pair or a Pro controller. (obviously there may be more controller types than this eventually, but accounting for everything would be impractical. Those two are common though)
This isn't critical, but it would be helpful in some cases.The final thing I'd like to bring up is the colour feature.
While I'm very happy it exists at all, I'm not sure it works as intended. You'll note in the raw data that there are two RGB values for one 'controller'. - obviously, most of the time FUZE logically groups two joycons as being a single controller, so this makes sense.But notice the colour patches; these are those same colour values drawn using rectangles.
There's a cyan/blue patch and some kind of darker shade of the same thing.now look at my second screenshot here; It's the switch's controller management UI. It shows several things.
Firstly, all the controllers I own (one pair of joycons, and a pro controller - technically I have others, but these aren't official controllers so they probably don't contain colour data)It also shows that the Switch can determine the colour of attached controllers; both joycons AND Pro controllers (you see the controller is black with neon green and neon pink grips - that's a splatoon themed controller. But crucially, the system 'knows' this somehow and can display it correctly - whether this is still just 2 colour values or maybe 3, I don't know.)
So, looking at the screenshot you can see I have the Neon Blue/Red controller pair, and a PRO controller that (assuming the black part isn't also a known value) registers to the switch system as Neon Green/pink.
So, what about FUZE? Well, you see the result in the first screenshot. For the Blue/Red pair, it returns two shades of blue.
But it's kinda self-evident (to me) that what it should be returning, is one blue value, and one red value.
(either that, or there are 4 colours returned, and the second shade of blue is an accent colour. In which case the returned values for the red controller are missing entirely)Furthermore, if you connect a single joycon somehow, and read values from that, you get two shades of grey, and if you read the data for the Pro controller you ALSO get two shades of grey.
I assume this is some kind of default return value, if FUZE can't work out actual values?
But as you can see, the Switch UI is fully capable of displaying colours for a Pro controller, so it seems reasonable that FUZE would be able to as well.
And a single joycon by itself STILL has a colour either way, so you wouldn't expect to get 'grey' just because you've got a single joycon, instead of two.This seems really unlikely to be the intended behaviour, so I assume this constitutes a bug.
Anyway, thanks for taking the time to read all this.
I know it's a lot to take in, but I thought it'd work better if I mention it all at once, rather than spread through many posts.
Especially since I discovered all these things while working on a single test program.Even if some of these aren't bugs, I thought I'd mention them anyway, just to bring them to someone's attention one way or another.
-
@KuraIthys Wow, thanks for your detailed post!
I expect that you will get a more detailed response from the devs but as I am responsible for most of the help I thought that I would give you a quick response now:
We are aware that some of the controls are undocumented at present. An issue has been raised for this
I believe that some more string handling functions are in the pipeline but you also might want to look at this:
-
Ah. That's very helpful. And it shows that the string functionality for accessing substrings DOES exist.
which is a relief to me, because it's a very common thing you end up needing to do with strings.So, I can cross that one of the list. (well, aside from the documentation issue).
I did expect some of these things are just my lack of knowledge about Fuze's full feature set. But it still seemed worth mentioning.
-
@KuraIthys Very interesting post. It looks like the colors are ok by now
But I noticed, that they are not updated during dis- and re-connection.
