CJS Framework – Work in progress

After a long research for the perfect crossplatfrom multitarget language i didn't find anything that satified my expectations. I wanted to create c++, c#, java and javascript projects around a set of basic classes, functionalities and serialization to import/export files in a custom format.

In a first evaluation, Haxe language with json seemed to be the perfect choise, but after using it i realized that the code produced for the languages of my interest is too heavy, bad indented and hard to reuse in the context of a specific project. That's because Haxe and the other multitarget languages are studied to produce a final result (i.e. a game) and not a source code that is easy to understand for the human reader or to include in other part of a wide project. For instance, if i create an Hello world example with haxe and convert it to c#, it will generates alot of useless .cs files in a well structured (and unreadable) project that you can build and run at first attempt, but that is barely readable and hard to reuse into other clean c# projects. If you are thinking of creating a library in haxe to include the source code into your projects, it's better if you change idea: the final result of your cpp, c#, java or javascript projects may end to a gigantic mess.

For this reason, I decided to create from scratch a new framework that is called CJS Framework, which stands for CppJavaScriptSharp Framework. It's mainly a full bridge between these four languages (but it's not improbable that it will support other languages in the future). The basic idea is very simple: make a set of classes that are useful in a project and a serialization system that easly load objects regardless of the used language. For instance, we can decide to take advantage of the .NET framework and produce an editor in C# to manipulate the maps of a game that will be programmed in javascript and it will run on facebook or the web browser. Or a server application in c++ that communicates in a binary format with a client application for html5 coded in javascript. The framework is projected to satify easly all these tasks and much more.

1 Basic Principles of the implementation

1.1 Introduction

The basic principle is to support features on languages avoiding the reinvention of the wheel if it's not necessary. For example, on c++ there will be a basic framework that will handle strings and objects, meanwhile on c# that part is skipped because it is already implemented on the native language. The most important thing is to have a set of classes that will facilitate the exchange of data. This data could be formatted into xml/json to be readable or modifiable by human beings, or binary like google protofub to be interchangable or raw to be light and optimized in terms of bandwidth at the expense of interchangeability. For example, immagine to serialize million of opengl calls from a client to a server, you cannot do it in xml, json and neither in protobuf also if it's a binary format, the best way for doing it is in raw format. But also if you have to handle a simple raw format, a framework is still necessary to handle 'obscure' stuff like the endiannes, multithreading, synchronization issues or the queue. If you decide to reprogram everything everytime you need it, you risk to injure your mental wellness with never ending session of debugging.

For these and other reasons, I decided to program a crossplatform multitarget language framework from scratch. It's not an easy task to do, but it's useful for my personal interest and maybe a great starting point for the birth of a new language. Anyway you must not think about it like the umpteenth framework on the market but much more like a programming phylosophy: make it easy, make it portable, make it readable anywhere. After 20 years of programming experience, i didn't find anything satisfactory from this point of view. I'm not trying to make the difference, to make viewers or to take attention, i'm finding the best way to code a project without GB of already existing frameworks with tons of redundant code. I really hope to succeed this time.

1.2 The minimal framework for c++

At first glance I was convinced that Poco Foundation libraries were the right choise to guarantee a minimal framework for the c++ base of the cjs framework. Studying the poco foundation project and taking a look at its internal architecture, i found it too broad (the full compilation of the library may take a gigabyte!). Moreover, the idea to have an heavy dependency to include into the framework does not thrill me a lot. The other alternatives were boost, qt, apache, but they were too heavy to be a part of a minimal and lightweight part of the framework. The final decision was the hardest one: code everything from scratch, without external dependencies. Also if this decision could be painful at a first time, in my opinion it is the best to avoid gigabytes of external dependencies and huge executable files, if the project is programmed in c++. Thanks to this important decision, the c++ part of the cjs framework will be lightweight, easy to build and transport. Anyway, if you are looking for advanced features, you can include poco foundation, qt or boost though.

The c++ framework is based on standard template library, so part of the classes (like strings, vectors, lists, maps) will be implemented starting from it. I decided to not use C++0x, C++11 or C++13 because it could restrict the range of portability of the framework, the readability of the code, the uselessness of the context and finally because I really don't like it. So old classic C++ and that's it, you don't need anything else.

1.3 Support UTF-8 and Unicode

C++

To support utf-8 i decided to create a class String that inherits from std::string and that handles utf-8 format along common 8 bits characters. A conversion will be performed to get a unicode string where required:

HANDLE loadLibrary(const cjs::String &filename)
{
    std::wstring filenamew;
    return LoadLibraryW(filename.getData(filenamew));
}

HANDLE hmod1 = loadLibrary("test.dll");
HANDLE hmod2 = loadLibrary(L"test.dll");

C#, Java, JavaScript

Fortunately, the support of unicode and utf-8 is native on these languages.

1.4 Instance object from class name

C++

template<typename T> cjs::Object *createInstance() { return new T; }
 
typedef std::map<std::string, cjs::Object *(*)()> instanceMap;
 
instanceMap map;
map["DerivedA"] = &createInstance<DerivedA>;
map["DerivedB"] = &createInstance<DerivedB>;
 
...
 
cjs::Object *newObject1 = map["DerivedA"];
cjs::Object *newObject2 = map["DerivedB"];

C#

var newObject1 = Activator.CreateInstance(namespaceName, "DerivedA");
var newObject2 = Activator.CreateInstance(namespaceName, "DerivedB");

Java

Class<?> newObject1 = Class.forName("DerivedA");
Class<?> newObject2 = Class.forName("DerivedB");

JavaScript

var newObject1 = window['DerivedA'];
var newObject2 = window['DerivedB'];

1.5 Boxing and unboxing

C++

cjs::Object *maybeAString = new cjs::Boxed<cjs::String>("Hello World!");
cjs::Object *maybeAVector = new cjs::Boxed<cjs::Vector3f>({0, 0, 0});
cjs::Boxed<cjs::String> *boxedString;
 
boxedString = dynamic_cast<cjs::Boxed<cjs::String> *>(maybeAString);
if (boxedString) {
   cjs::String &testString = boxedString->getValue();
}

cjs::Boxed<cjs::Vector3f> *boxedVector3f;
boxedVector3f = dynamic_cast<cjs::Boxed<cjs::Vector3f> *>(maybeAVector);
if (boxedVector3f) {
   cjs::Vector3f &testVector3f = boxedVector3f->getValue();
}

cjs::ListObject *testList = new cjs::ListObject();

testList->pushBack(new cjs::Boxed<cjs::String>("String1"));
testList->pushBack(new cjs::Boxed<cjs::String>("String2"));
testList->pushBack(new cjs::Boxed<cjs::Vector3f>({0,0,0}));

C#, Java, JavaScript

Not yet decided. The boxing is native on these languages, but it may require some efforts to make it compatible with the serialized data format.

1.6 Serialization

C++

class Example
{
...
    void readSerialized(Serializer *p_serializer);
    void readSerializedContent(Serializer *p_serializer);

    void writeSerialized(Serializer *p_serializer);
    void writeSerializedContent(Serializer *p_serializer);

private:
    String m_name;
    uint32_t m_index;

    float m_mat4x4[16];

    Vector3f m_position;
    Vector3f m_axis[3];

    float *m_dataPoints;
    uint32_t m_numOfPoints;

    Vector3f *m_dataVectors;
    uint32_t m_numOfVectors;

    Example *m_node;

    void **m_objects;
    uint32_t m_numOfObjects;
};

#pragma pack(push, 1)
class Vector3f
{
...
    void readSerialized(Serializer *p_serializer);
    void readSerializedContent(Serializer *p_serializer);

    void writeSerialized(Serializer *p_serializer);
    void writeSerializedContent(Serializer *p_serializer);

private:
    float m_x;
    float m_y;
    float m_z;
};
#pragma pack(pop)

void Example::readSerializedContent(Serializer *p_serializer)
{
    // Read a single string (int tag, String name, int count, T *variables)
    p_serializer->readStatic<String>(1, "name", 1, &m_name);

    // Read a single value (int tag, String name, int count, T *variables)
    p_serializer->readStatic<uint32_t>(2, "index", 1, &m_index);

    // Read a static array of values (int tag, String name, int count, T *variables)
    p_serializer->readStatic<float>(3, "mat4x4", 16, m_mat4x4);

    // Read a single object (int tag, String name, int count, T *variables)
    p_serializer->readStatic<Vector3f>(4, "position", 1, &m_position);

    // Read a static array of objects (int tag, String name, int count, T *variables)
    p_serializer->readStatic<Vector3f>(5, "axis", 3, m_axis);

    // Read a dynamic array of values (int tag, String name, int *count, void **variables)
    p_serializer->readDynamic<float>(6, "dataPoints", &m_numOfPoints, &m_dataPoints);

    // Read a dynamic array of objects (int tag, String name, int *count, void **variables)
    p_serializer->readDynamic<Vector3f>(7, "dataVectors", &m_numOfVectors, &m_dataVectors);

    // Read a single instance of an object (int tag, String name, void **object, void *(*instanceFunction)(const String &))
    p_serializer->readSingleInstance<Example>(8, "node", &m_node, NULL);

    // Read a list of multiple instances of objects (int tag, String name, int *count, void ***objects, void *(*instanceFunction)(const String &))
    p_serializer->readMultipleInstances<Example>(9, "objects", &m_numOfObjects, &m_objects, NULL);
}

void Vector3f::readSerialized(Serializer *p_serializer)
{
    readSerializedContent(p_serializer);
}

void Vector3f::readSerializedContent(Serializer *p_serializer)
{
    p_serializer->readStatic<float>(1, "x", 1, &m_x);
    p_serializer->readStatic<float>(2, "y", 1, &m_y);
    p_serializer->readStatic<float>(3, "z", 1, &m_z);
}

 

void Example::writeSerializedContent(Serializer *p_serializer)
{
    // Write a single string (int tag, String name, int count, const T *variables, int rowLength)
    p_serializer->writeStatic<String>(1, "name", 1, m_name, 1);
    
    // Write a single value (int tag, String name, int count, const T *variables, int rowLength)
    p_serializer->writeStatic<uint32_t>(2, "index", 1, m_index, 1);
    
    // Write a static array of values (int tag, String name, int count, const T *variables, int rowLength)
    p_serializer->readStatic<float>(3, "mat4x4", 16, m_mat4x4, 4);

    // Write a single object (int tag, String name, int count, const T *variables, int rowLength)
    p_serializer->writeStatic<Vector3f>(4, "position", 1, m_position, 1);
    
    // Write a static array of objects(int tag, String name, int count, const T *variables, int rowLength)
    p_serializer->writeStatic<Vector3f>(5, "axis", 3, m_axis, 1);

    // Write a dynamic array of values (int tag, String name, int count, void *variables, int rowLength)
    p_serializer->writeDynamic<float>(6, "dataPoints", m_numOfPoints, m_dataPoints, 6);

    // Write a dynamic array of objects (int tag, String name, int count, void *variables, int rowLength)
    p_serializer->writeDynamic<Vector3f>(7, "dataVectors", m_numOfVectors, m_dataVectors, 1);

    // Write a single instance of an object (int tag, String name, void *object, String (*lookupFunction)(void *))
    p_serializer->writeSingleInstance<Example>(8, "node", m_node, NULL);

    // Write a list of multiple instances of objects (int tag, String name, void **objects, String (*lookupFunction)(void *))
    p_serializer->writeMultipleInstances<Example>(9, "objects", m_numOfObjects, &m_objects);
}

void Vector3f::writeSerialized(Serializer *p_serializer)
{
    writeSerializedContent(p_serializer);
}

void Vector3f::writeSerializedContent(Serializer *p_serializer)
{
    p_serializer->writeStatic<float>(1, "x", 1, &m_x);
    p_serializer->writeStatic<float>(2, "y", 1, &m_y);
    p_serializer->writeStatic<float>(3, "z", 1, &m_z);
}
Example example;

IOStream *ioStream = new IOStreamFile("example.xml");

WriterXml *writerXml = new WriterXml();
writerXml->setOutputStream(ioStream->getOutputStream()->addRef());

Serializer *serializer = new Serializer();
serializer->setWriter(writerXml->addRef());

serializer->beginVariable(1, "example");
example.writeSerialized(serializer);
serializer->endVariable();

serializer->finish();

writerXml->release();
ioStream->release();
serializer->release();

output:

- xml

<example class="Example" name="test" index="0">
    <mat4x4>
        1 0 0 0
        0 1 0 0
        0 0 1 0
        0 0 0 1
    </mat4x4>

    <position x="10" y="20" z="30" />

    <axis>
        <elem x="0.5465353" y="0.435839" z="0.698445" />
        <elem x="0.5465353" y="0.435839" z="0.698445" />
        <elem x="0.5465353" y="0.435839" z="0.698445" />
    </axis>

    <dataPoints>
        0.5465353 0.435839 0.698445 0.238954 -0.965926 -0.490903
        0.5465353 0.435839 0.698445 0.238954 -0.965926 -0.490903 
        0.5465353 0.435839 0.698445 0.238954 -0.965926 -0.490903
    </dataPoints>

    <dataVectors>
        <elem x="0.5465353" y="0.435839" z="0.698445" />
        <elem x="0.5465353" y="0.435839" z="0.698445" />
        <elem x="0.5465353" y="0.435839" z="0.698445" />
    </dataVectors>

    <node class="Example" name="Empty" index="0" />

    <objects>
        <elem class="Example" name="Empty1" index="0" />
        <elem class="Example" name="Empty2" index="0" />
        <elem class="Example" name="Empty3" index="0" />
    </objects>
</example>

- json

example:{class:"Example", name:"test", index:0,
    mat4x4:[
        1, 0, 0, 0,
        0, 1, 0, 0,
        0, 0, 1, 0,
        0, 0, 0, 1
    ],
    position:{x:10, y:20, z:30},
    axis:[
        {x:0.5465353, y:0.435839, z:0.698445},
        {x:0.5465353, y:0.435839, z:0.698445},
        {x:0.5465353, y:0.435839, z:0.698445}
    ],
    dataPoints:[
        0.5465353, 0.435839, 0.698445, 0.238954, -0.965926, -0.490903, 
        0.5465353, 0.435839, 0.698445, 0.238954, -0.965926, -0.490903, 
        0.5465353, 0.435839, 0.698445, 0.238954, -0.965926, -0.490903
    ],
    dataVectors:[
        {x:0.5465353, y:0.435839, z:0.698445},
        {x:0.5465353, y:0.435839, z:0.698445},
        {x:0.5465353, y:0.435839, z:0.698445}
    ],
    node:{class:"Example", name:"Empty", index:0},
    objects:[
        {class:"Example", name:"Empty1", index:0},
        {class:"Example", name:"Empty2", index:0},
        {class:"Example", name:"Empty3", index:0}
    ]
}

- raw

"Example", "test", 0, 
1, 0, 0, 0,
0, 1, 0, 0,
0, 0, 1, 0,
0, 0, 0, 1,
10, 20, 30,
0.5465353, 0.435839, 0.698445,
0.5465353, 0.435839, 0.698445,
0.5465353, 0.435839, 0.698445,
18,
0.5465353, 0.435839, 0.698445, 0.238954, -0.965926, -0.490903, 
0.5465353, 0.435839, 0.698445, 0.238954, -0.965926, -0.490903, 
0.5465353, 0.435839, 0.698445, 0.238954, -0.965926, -0.490903,
3,
0.5465353, 0.435839, 0.698445,
0.5465353, 0.435839, 0.698445,
0.5465353, 0.435839, 0.698445,
"Example", "Empty", 0,
3,
"Example", "Empty1", 0,
"Example", "Empty2", 0,
"Example", "Empty3", 0

 

Gianpaolo Ingegneri
Copyright @ 2015 - All right reserved