Draft: Designing a Data Flow Language - Type System

Renato Pereira

In sequence to my previous post where I talk about the pipe expressions, here I will explore some ideas for a type system.

My goal designing this type system is forcing synergy between it and the pipe expressions. I have already came to conclusion that I will use a dynamic an weak typing, but I also have in mind some possibilities for the following key points:

Dynamic and Weak Typing

As I have already discussed, this language will have a dynamic weak typing because I don’t have enough skills (and time) to implement a full-featured statically type system, which would involve generics and inferaces and polymorphism and etc. Moreover, this is a scripting language for prototyping and fast interaction, it should not be used for complex and risk applications.

The weak typing part is about Stream convertion and some ideas related to error handling.

Errr Handling

Before diving into my decisions for the new language, I would like to digress a bit and give some notes about error handling in different languages.


My first reference is the classic try-catch block. This constructions is the most known error handling technique in programming, and it sucks! The try part opens a new scoped block which forces us to declare a variable outside if we want to use it later. If you want to ignore the error, you still have to use the try-catch, otherwise you program will explode in a runtime error.

let file;
try {
  file = fs.Open("file");
} catch (error) {
  return error;

let data;
try {
  data = JSON.parse(file);
} catch (error) {
  return error;

return data;

Golang offers an interesting alternative: handling the error as part of the result. Any function that may “throw an error”, just return it in a tuple with the results, forcing the user to acknowledge (treating or ignoring) the error explicity.

file, err := os.Read('file')
if err != nil {
	return nil, err

var data Data
err := json.Unmarshall(file, &data)
if err != nil {
	return nil, err

return data, nil

The part that I don’t like is not what you see in the code, is what you don’t see. Errors can still happen in runtime (for example, nil pointer exceptions), and when they happen, your program panic and to avoid it, you gotta add the ugly defer recovers to avoid problems in isolate parts.

Functional languages usually handle errors using Optional (monads, maybe, union, whatever) types, which also make error handling explicit. The side effect is that you can use this pattern for eliminating null types entirely. I lack the knowledge to know if these languages also have the panic structure similar to go.

case File.read(file_path) do
	{:ok, file} ->
		case Jason.decode(file) do
			{:ok, data} -> {:ok, data}
			{:error, _} -> {:error, :json_decoding_failed}
	{:error, _} -> {:error, :file_read_failed}

The Elixir’s snippet above also shows another cool feature that synergizes with this strategy: pattern matching. Languages that borrow ideas from functional world such as rust, have similar error handling pattern:

// file.read_to_string(&mut contents)?; <-- the ? operator returns the error
// immediately

let file_content = match fs::read_to_string(file_path) {
		Ok(content) => content,
		Err(err) => {
				eprintln!("Error reading file: {}", err);

let data: Data = match serde_json::from_str(&file_content) {
		Ok(data) => data,
		Err(err) => {
				eprintln!("Error parsing JSON: {}", err);

Finally, languages such as Zig seek to handle errors in a more traditional way with a simple try expression.

file, err = try readFile();
if (err.message != null) {
		log.error("Error: {}", err.message);

// Attempt to parse JSON and handle errors
data = try parseJSON(file);
if (err.message != null) {
		log.error("Error: {}", err.message);

My previous language, SHT, introduces a pattern which I baptized of “wrapping system” (I also like the “Gift 🎁 System”). Which is a mixture of the patterns above:

file = os.Open('file')? # wrap into a Maybe type
return if file! 				# unwrap the err
return json.Parse(file) # it will have same effect as the above

The new part is the unwrap effect: a! would be equivalent to { err := a.Err(); a = a.Value; err }. In other words, the unwrap expression forces the variable a to assume its value (either the content or the error) and the expression returns the error, so it can be handled separetely.

I like the shortcut if file! as err { ... handling err here ... } but I don’t like the variable side effect. I also don’t like functions returning different types, which can lead to serveral new errors.

My Decision

If this were a static language, I would force any function that throws an error to return a Maybe object. Since it is not, I won’t mix type returns with errors and I will just throw the error instead, forcing the interruption. So raise keyword added.

I will still be using the Maybe type, but for catching the error. I will use the SHT gift pattern for this. Thus ? and ! operators confirmed.

There is only one difference from SHT system, the ! operator will return (err, val) instead of just the error. The tuple system will handle the rest (see the tuple section below).

I still unsure about the unwrap changing the variable, but I will keep this behavior to keep the language concise.

Null Values

Null values are traditionallly used to denote invalid, missing or unitialized values. They may originate from external sources such as JSONs or internal sources such as variable declarions and errors.

Internal sources its easy to handle, we can just use Maybe to represent the missing or invalid values, and initiatializations should enforce default values (which could also may be Maybe in some cases).

External sources can be used as input or output of our program. For example, I can read a json data with null or I can write a json data with null. Same thing for databases, csvs, http requests. The thing is, we can’t just remove null.

dict = json.Load(`
  "a": null

dic['a'] -- present or not?
         -- maybe, err or constant?
         -- return err if present?
         -- throw err if not?

Presence is important because it may have different meaning than undefined (vide JavaScript rcosystem) and the output may require it. Thus it comes to how to represent null values.

We could represent null as a special constant NULL, but then, what’s the point of removing it?

We could represent null dorectly as an error. But then we haventhe same discussion as the error throws/returns.

Representing null as Maybe will be the most consistent way.

Functions and lambda

I don’t have much thoughts about common functions. High order is no brainer here. My focus is how to create lambdas.

Let’s get back to the pipes, the main feature of the language. I tried a LOT of different syntaxes over the past few days. The most interesting format I have found was:

| <func> <param1>, <param2>, ...

-- which is equivalent to

<func>(Stream(<value>), <param1>, <param2>, ...)

-- examples:

| map ip: http.Ping(ip)


value := math.fibonacci()
| takeWhile x: x < 4000000
| filter    x: x%2 == 0
| sum
| to        Number

I’ve also tried different syntaxes for lambdas. => like javascript, where, lambda :, etc. The best one was just :.

double   := x: x*2
distance := (x, y): math.Sqrt((x - y)^2)
two      := :2

Tuples and pattern matching

I like tuples, they are useful. But I messed up really bad in SHT implementation. This time my approach will be:

fn one() { return 1 }
fn two() { return (1, 2) }
fn three() { return (1, 2, 3) }

a0 := one()           -- a0 = 1
a1 := two()           -- a1 = 1
a2, b2 := two()       -- a2 = 1, b2 = 2
a3, b3, c3 := three() -- a3 = 1, b3 = 2, c3 = 3

a, b := one() -- error!

Lambda function parameters ((a, b): a + b) could be represented as tuples internally too. Maybe even function calls (f(1, 2))?. Pattern matching take advantage of this:

match (i%3, i%5) {
  (0, 0): println('FizzBuzz')
  (0, _): println('Fizz')
  (_, 0): println('Buzz')
  (_, _): println(i)

Notice that match can be seem as a block of lambda functions! The same could be applied to pipes:

| map ip: ip.Split('.')
| match (127, _, _, _): println('Blocked')

Some languages use _1 to reference the matched element after, which may be useful.

Custom data types

Too be honest, this is my least favorite topic, I just want something simple enough. Specially if I’m not working with statically typed language. The only concerns is how to handle initialization without null:

data GraphNode {
  Id     = newId()
  Weight = 10
  Parent = Maybe()

  fn Print(this) {
    print("%s", this.Id)

    parent := this.Parent
    while parent.Ok() {
      print("-> %s", parent.Id)
      parent = parent.Parent

root := GraphNode { Id = 'root' }
child := GraphNode { Parent = root }



Up to now, I can’t think on any approach to fix this, other than immutability or operator override (maybe override the assignment = operator).