Writing C++ Modules For Feral

Published in

Level Up Coding

9 min readApr 21, 2020

Extending the Feral programming language using C++

Build script for the module created in this article

In my last article, I introduced the Feral programming language. This time around, we’ll create C++ extensions for it! These extensions are building blocks for adding functionality and library support for the language. Usually, a generic task can be implemented in Feral itself, but writing it in C++ is especially useful when the task requires native performance.

For the purpose of this article, we’ll be implementing a C++ module, containing a Feral variable and function, to demonstrate how to properly utilize the available features to our advantage.

Any doubts, questions, etc. can be asked either on GitHub or on Feral’s discord server (links at bottom). 🙂

Concepts

In Feral, a C++ extension works as follows:

Load the extension at runtime using the mload() function in our Feral script. Internally, this will open the given dynamic library (the module) and call a special function in it called init_<module_name>() (created in the module using the INIT_MODULE(module_name) macro).
Inject/Insert the required variables and functions into Feral runtime through the init_<module_name>() function.
The variables and functions are now available to be accessed from the Feral source.
If there are dynamically allocated variables, they can be deallocated using the optional deinit_<module_name>() function (created using DEINIT_MODULE(module_name) macro). An example for this is the Feral-Lang/Feral-Curl module.

That’s fundamentally it! Easy enough, I think… I hope at least? 😁 🙈

Writing Our Module — Creating a Variable

For a simple module with explanations and complete source code, check out ImMaax’s repository: ImMaax/Feral-HelloWorld.

wOkay, let’s go for the fun stuff! First of all, we need to create a directory structure for our module. Feral comes with a build system which, when provided with a particular directory structure and a build script (build.fer), will build and install your Feral C++ module for you. Simple as that.

The directory structure is as follows:

<a_module_directory>/
    ├─ include/
    ├─ src/
    └─ build.fer

The src/ directory will contain the C++ source files and the include/ directory will contain the .fer import script. Note that neither of them are necessary. We will create our .cpp module source file in the src/ directory. For now, let’s name it learn.cpp (yep, I know — so creative!).

Now our directory structure looks like this:

<a_module_directory>/
    ├─ include/
    ├─ src/
    │   └─ learn.cpp
    └─ build.fer

Now, we are ready to dive in the (gorgeous) C++ source code.

Since all the required declarations are present in the feral/VM/VM.hpp header file in the PREFIX_DIR of your Feral installation (more on that here), we will have to include that file in our source.

We also use the INIT_MODULE macro which provides an entry point to our module for Feral. Our source file will look something like this:

Our barebones C++ extension source file

Note that the module name must be the same as source file name (minus .cpp part), including the alphabet capitalization.

Let’s create a source local variable — say pi (3.14157).

Since the variable is source local (available only in the Feral source file which called mload() to load this module), we need to get the currently opened Feral source file and then we can add the variable to it.

To fetch the current source, we basically get the top element in the source file stack of the Feral VM, and store that element in a var_src_t* type variable.

This sums up to: var_src_t * src = vm.current_source();.

To actually insert the variable in source, we have to call the add_native_var function on the src pointer. The entire statement for inserting pi in the source is:

src->add_native_var("pi", make_all<var_flt_t>(3.14157, src_id, idx));

Let’s break this down.

From the innermost side, we see make_all<var_flt_t>. In essence, var_flt_t is a Feral type for storing floating point values. make_all is a function that allows us to initialize the type (here var_flt_t) with required values.

The arguments for var_flt_t are:

1. float: the floating point value to be created
2. size_t: the current source ID
3. size_t: the current source location indexFor the first argument, we enter the pi's value - 3.14157.
src_id and idx are provided to us through the INIT_MODULE macro itself.

The src_id and idx variables are crucial as, on occurrence of an error, they are used to show the source file, line, and column where it occurred.

The first argument to src->add_native_var() is the name with which our variable will be available in Feral (here pi).

Well, that’s it! We created our first variable for Feral in our extension! The entire source code should look like:

Our C++ extension source file with the pi variable

Now, what’s left to do is create our build script (build.fer) and install this module for us to test. The build script is as follows:

let sys = import('std/sys');
let builder = import('std/builder');let build = builder.new().make_dll();
build.add_src('src/learn.cpp');sys.exit(build.perform('learn'));

The only two things we need to note in this script for now are: build.add_src('src/learn.cpp'); and sys.exit(build.perform('learn'));.

These are our source file location and the build name respectively. Just remember to name these according to your file location and module name.

After that, we will install our module using the feral install command. But first, make sure that Feral’s module directories are initialized using feral init command. For me, the output of feral install is:

Yay! Time to test this out!

Just create a script — say test.fer, import the io module and mload() our custom module in that file, and use the io.println() function to see if we can print the value pi. The code for that is as follows:

let io = import('std/io');
mload('learn');io.println('Value of pi is: ', pi);

You’ll get the following output:

Value of pi is: 3.14157000000000019568

Yay!! We made our first C++ extension!!

But, it isn’t complete yet. Notice the mload() instead of import()? Yea, we don’t want that. The reason being that the symbols in it cannot be stored in a variable (like io), which means it is unusable outside this script.

So, we want to wrap this mload() call in a Feral script which we will import as required, instead of using mload() again and again.

To do that, create a Feral script — say learn.fer in the include/ directory, and just write the mload() function call in that script:

mload('learn');

Now, do a feral install again and boom! we are done with creating our simple module!!

To use it, instead of calling mload(), we will call import() and store it in a variable which will contain our pivariable. Our final test script will look like this:

let io = import('std/io');
let learn = import('learn');io.println(learn.pi);

Woohooo! Our first C++ extension for Feral! Isn’t it awesome?! 🤩😍

Time to create a C++ function for Feral!

Writing Our Module — Creating a Function

A Feral function’s C++ signature is specific. We must use that signature to properly create a Feral function. The signature is:

var_base_t * func(vm_state_t &, const fn_data_t &);

The function name (here func) can be whatever you want. The function must have 2 arguments:

1. vm_state_t&: virtual machine state - provides access to the VM
2. const fn_data_t&: the function call information - mainly arguments, keyword arguments, src_id, and idx

And, the function must return a Feral variable object pointer (var_base_t*).

Well, let’s create a function that returns length of a string!

We know, in C++, the length of string is returned by the std::string::size() member function. We can leverage that. And of course, Feral has a wrapper for C++’s string type. You guessed it — var_str_t.

We will create a function len() which will take the string as parameter, and return the length of that string. The code for that is:

Length of string (part 1)

… Yea, we should break it down.

The first line is of course, the function signature we discussed about — the function name here is len. Inside the function body, we fetch the src_file object of the source file currently being executed — this is done to properly produce errors. Then, we check if the type of our first argument (fd.args[1]) is string (VT_STR) or not. If it is not a string, we use the vm.fail() function to show an error on the position of our first argument (fd.args[1]->idx()) and return nullptr.

Note that the first argument is not fd.args[0] because that is reserved. We’ll get to that later. Also, when we return nullptr it actually means that the function has failed and Feral will stop executing after that.

Finally, we typecast our base class object fd.args[1] to var_str_t type using the STR() macro, then we get() the wrapped std::string object and use the size() member function to fetch the length of it, after which, we wrap that value to a var_int_t and return it. That’s it!

You must have noticed that this time we used make<>() function instead of the previous make_all<>() that we used. The difference between them is that we do not set the src_id and idx when using the make function which is what we want here because after the function call, Feral sets that value by itself.

There we have it! Our own Feral function! However, we also need to let Feral itself know about this function’s existence. For that, we’ll add it to Feral’s currently executing source file (similar to how we added our variable before).

We will, therefore, use the add_native_fn() member function of our src variable in the INIT_MODULE block.

The arguments for that function are:

1. string: the name for function which will be visible in Feral
2. nativefnptr_t: the function we created in the C++ source
3. int (optional): number of arguments required by the function
4. bool: does the function use variadic arguments

Since we want our len function to be called len in Feral, which takes 1 argument and is not variadic, the function call will be as follows:

src->add_native_fn("len", len, 1);

Yep! We’re done!! We can try our function now! 😁

Let’s feral install again to update our module and write the following in test.fer:

let io = import('std/io');
let learn = import('learn');io.println(learn.len("some string"));

When we run it, we’ll see the output 11 which is the length of some string. Yay! it worked!! 🤩

Although, wouldn’t it be better if we could do something like… "some string".len()? Let’s do that!

To create a member function (or correctly, type bound function) we need to bind the function to a specific type (here VT_STR) and change a little bit of our function body.

For the function body, remember we had a reserved fd.args[0]? Well, it is actually the variable that contains the object that called this function. In other words, if we do "some string".len(), fd.args[0] will contain "some string" (similarly, when we did learn.len() learn object was contained by it).

Also, since this function will be bound to the string type, we will not have to check if the argument is a string like we did before!

Performing those changes, our new function becomes:

Length of string part 2

Note that we also removed src_file since it was no longer required.

This looks small and cute, doesn’t it! 😍

Finally, updating the function declaration in INIT_MODULE, we replace the src->add_native_fn() call with vm.add_native_typefn(). The arguments for this function are:

1. int: type to bind the function to
2. string: the name for function which will be visible in Feral
3. nativefnptr_t: the function we created in the C++ source
4. int: number of arguments required by the function
5. size_t: the current source ID
6. size_t: the current source location index

Since this is a going to be a member function of string type, we no longer have to provide any argument to it. Therefore, the argument count will now be 0 instead of the previous 1. Therefore, the final function call becomes:

vm.add_native_typefn(VT_STR, "len", len, 0, src_id, idx);

Now, use feral install to install updated module, and change the test.fer to use "some string".len() instead of learn.len("some string").

The code becomes:

let io = import('std/io');
let learn = import('learn');io.println("some string".len());

Note that the learn. prefix is not required anymore.

Execute the code and we’ll see the output 11 which is the length of some string. Perfect!!

The entire source code for our C++ module is:

Full module source

Conclusion

This was a basic article/tutorial on how to create C++ extensions for Feral. I hope it was informative and fascinating. 😁

If you want more examples look in the Feral-Lang/Feral-Std repository (the standard library for Feral). The relevant links are given in the section below.

As always, all questions, ideas, suggestions, and thoughts are pleasantly welcome and greatly appreciated.

Thanks a lot for reading, and have a great day. Until next time! ❤️