Memory management, variable assignment, argument usage

[IANRASE]

So a couple thoughts on memory management;

First off, different resources are freed with different commands, which is confusing.
i.e. for images you have freeImage, but for shapes it's deleteShape and maps are unloadMap while sprites, objects, and lights are removeSprite, removeObject, and removeLight.
Meanwhile, there is no way to free space allocated by createTerrain, loadAudio or loadModel.

Additionally, all of these things could be managed automatically by freeing them when the last remaining reference is reassigned or goes out of scope.

The programmer shouldn't have to worry about the rendering engine still having a reference to an object because from his/her perspective, they have the only reference to it - reassigning the last reference towards an object that still has a hidden reference by the renderer should either cause an error or remove the reference held by the renderer.

If the programmer must manually manage these things, then there needs to be 2 things:

• The documentation and tutorials must mention which functions cause unmanaged memory allocation, and these functions' reference pages should mention the corresponding de-allocation function in their overview and examples.

• There must be a clear distinction between reference duplication vs value duplication when variable assignment or argument passing.

The other problem, which has been pointed out before but not yet as a topic, there is no indication of when variables are passed, returned, or assigned by reference or by value, and no way to ensure variables passed by reference aren't accidentally modified (or that they are modified, if that is the intent).

Further there is no indication of how assignment actually works for all types, for example, when I do var1 = var2, if var2 is a structure with other structures, arrays, strings, etc. as members, and some of those members also contain arrays and structures, does var1 become a deep copy of var2? i.e. for something like

someArray = [1, 1, 1, 1]
var1 = "what a neat variable"
var2 = [
    .struct1 = [
        .nothing = 0,
        .arry = someArray
    ]
    .imgs = [ 
        createImage(256, 256, false, image_rgb),
        createImage(256, 256, false, image_rgb)
    ]
]
var1 = var2
var1.struct1.arry1[3] += 1
setDrawTarget(var1.imgs[0])
box(0, 0, 100, 100, orange, false)

Will var1.struct1.arry[3] equal var2.struct1.arry[3]? Will var2.imgs[0] have an orange box on it?

Because all programs are confined to a single file and sharing is limited for the time being, the ambiguity aspect of the issue isn't as critical, but the overall lack of explanation or consistent demonstration will make it really hard for FUZE to work as a learning tool.

I know there are changes coming to parts of the help section along with some bugfixes, since there are currently no path notes indicating what specifically is being addressed, I wanted to post this here to make sure these specific issues can be addressed.

[Dave]

Very valid point about inconsistencies in the names of functions. In future we will look to add aliases for these functions, improving the consistency across the board. Thank you for bringing that one up.

On memory - there so far has not been a program made in FUZE which necessitates the use of the memory freeing functions that we know of. The chances of a user actually requiring them are very slim. Should it turn out that people are coming across problems because they cannot free memory, we will address it.

Out of interest, have you created a project in which you discovered this to be a problem? If so, please let us see it and we'll do our best to accommodate.

At the end of the day, FUZE has been designed to make learning to code as accessible as possible, by lowering the barriers to entry put in place by languages like C, etc.

[IANRASE]

I was working on sort of a 3D map maker where the player places 3D objects and can save/load their map, view selected object properties, get a snippet of what the code needed to create and place each object for their map as they currently have it would be, etc. Essentially its a tool help to take the trial and error out of object placement.

Basically the issue is that models and 3D objects are loaded and placed dynamically by the player, and the program maintains a few lists that could be memory intensive depending on how things are managed in the background and what is a reference vs what is a copied value.
Here is the basic structure of the program in pseudo code:

modelList = [ // list of models the player may load into the editor. This is filled in by the user before launching the program and is not to be modified by program code
    "fileName1",
    "fileName2",
    "fileName3"
]

struct Model // holds information relating to a model
    string fileName
    ModelHandle handle  // handle returned by loadModel(). Null until the first object using this model is created
    int instanceCount       // how many objects have been created with this model
endStruct

struct Object // holds information relating to an object
    string name       // display name for this object in UI. defaults to "[name of the model][instance count of model]"
    int basicShape  // 0 means object is not a basic shape (uses loaded model) or 1-7 for cube/sphere/pyramid/cone/cylinder/wedge/hemisphere
    Model mod        // the model for this object. May be changed to an array index in implementation to prevent "obj.mod = m" from creating an unnecessary copy of m
    Animation anim // animation info for the object
    vector position
    vector scale
    vector rotation
    vector point
    vector color
    int metallic
    int roughness
endStruct

mapper = [ // global structure for organizing all map and UI data
    .models = array<Model>[len(modelList)],
    .objects = List<Object>,
    .matPresets = List<MaterialPreset>, // list of preset material values
    .camPos = {0, 0, -20},
    .camTarget = {0, 0, 0},
    .currentObject = null, // of type Object. will probably be an array index in implementation
    .multiSelect = false, // when true multiple objects can be selected at once to be moved/scaled together
    .currentGroup = List<Object> // used only when multiSelect is true
    .gridSnapping = {} // value of 0 or 1 for X/Y/Z means snapping is on/off for that axis
    .gridSnapSize = {} // how large the grid should be for position snapping
]

Basically I'm worried that unless I'm careful and only store things like placeObject() or loadModel() handles in arrays and then refer to them always by array indexing, I won't have any way to control or guarantee memory leaks or null references won't happen. Since there's also no type checking or visible referencing, there's no way to know what kind of information any given variable might hold without looking at and accounting for all possible data flows - which is hard when the program gets very large.

While the player probably won't create so many objects that they will run out of memory, I am concerned of something like the player having a large map saved, then they load it, make some changes, then decide they don't like that so loading it again, then continuing to edit.. and so on until they do a load or create an object and it crashes due to some overlooked memory leak or data handling bug that causes them to lose their progress and is hard to hunt down since assignment often ambiguous and certain things that allocate memory can't be freed.

I'm sure it won't actually be a problem, and I haven't encountered anything like it that wasn't preventable yet, but it's just a constant concern as I'm writing code because I don't have a clear picture of what this function or that variable assignment specifically does, and the wording of function names, keywords, and their documentation often implies the existence of certain type schemes, OO structure, or reciprocal behavior as in other programming paradigms that aren't there.

It just feels to me that FUZE is a bit all over the place and it's new and we are currently lacking in experienced users and intricately detailed documentation.

[Zero Division]

@Dave @IANRASE I'd very much like to know when things are being passed by reference vs value. Seems like the documentation is the right place to put this information. Incidentally, do we have a way to check how much used/free heap and stack a program is using? That'd let me make some guesses at least.

[IANRASE]

I have actually run into several memory and assignment issues now with a different program I'm making (I gave up on the 3D map maker since I was spending more time on it than I would have saved using it as there is an official one planned anyway):

• declaring a 500 by 500 array causes the program to crash due to not enough memory (for 32bit int that's only 1 MB)
• you can assign non-int values to an integer array, but accessing them causes errors
  i.e.

array a[size]
a[0] = {1, 1, 0} // OK
print(a[0].x)     // causes error

• you cannot declare non-integer arrays unless you also initialize them
  i.e. to create an array of n vectors, you must do

a = [ {0, 0, 0} ]
for i = 0 to n loop
    a[i] = {0, 0, 0}
repeat

• there is no way to differentiate between local scope variables and global variables without wrapping them in structures (which is very cumbersome). This is only an issue because it's not clear when something is being re-assigned vs re-initialized as a new variable - which itself is only an issue because scoping and type-enforcement rules are not explicitly explained.
• Important operations and commands enforce type restrictions, but the programmer has no access to any type restrictions or type checking

Also on an unrelated note, there is a lack of reciprocal get functions for most of the functions for setting things for the screen, text drawing, or 3d objects.
for example it's impossible to determine what the current ink color is or the drawTarget or the current text size, unless you keep an additional variable and set it every time you change those things.

And although this has been mentioned before, I really can't stress how important either a switch or elseif statement is. Without it your control flow is either hard to read (and thus easy to screw up) or inefficient. Sometimes it feels like mutually exclusive conditionals are easier to read in assembly than what I've written on FUZE.

A continue would also be nice.

[Martin]

@IANRASE said in [Memory management, variable assignment, argument usage]

(for 32bit int that's only 1 MB)

I really, REALLY don’t want to get into the subject of memory management internally within Fuze because it’s outside of my control and not something that people should need to concern themselves with. BUT the above assumption is flawed.

There are overheads to managing and tracking variables internally so you can’t make simple math assumptions on the size of stuff internally.

[IANRASE]

@Martin I figured as much and should have stated that, what bothers me is the inability to utilize that 1MB of memory when I could do the same or similar within the other interpreted languages and scripts with similar limitations (which is silly of me since those languages aren't brand-new and don't also have a built in 2D and 3D graphics API all built within the restrictions of a Nintendo Switch application)

I don't know anything about what's under the hood in FUZE and I shouldn't be a back-seat driver anyway so I'll won't bring up memory management again unless I spot some glaring issue.

Most of my frustration will probably be alleviated as FUZE, the documentation, and the community all grow anyway - I'm just too excited and impatient.

[MikeDX]

@IANRASE said in Memory management, variable assignment, argument usage:

Most of my frustration will probably be alleviated as FUZE, the documentation, and the community all grow anyway - I'm just too excited and impatient.

I share this view. The version of fuze we have today is already a long reach from the original release. There is a lot of good stuff coming, I promise :) The version we have by Christmas, by Easter and by this time next year will be very different!