Prasanth Janardhanan

How to support custom Javascript scripting in Go Applications

Why will someone need a Javascript Parser, written natively in Go? Isn’t it a crazy, Architecture Astronauts solution that is looking for a problem? Not necessarily.

There was a time when applications allowed some kind of scripting to extend them and to make them fit into any workflow. For example VBScript for Microsoft office products. However, very few Web applications have the infrastructure to allow custom scripts inside them. There are a few that does support; one example is Google Apps Script.

Of course, most web platforms support REST APIs. However, it requires running a separate server that runs your custom scripts that can call the APIs. There are IFTTTs and Zapiers that can do integration with other similar services. However, none of those solutions can match the seamless integration possible by allowing custom scripting inside the web app itself.

Imagine you have an email service like Gmail or Hey. Let’s say your users want more control over their email workflow by using their own scripting. For example :


myemail.RegisterHook("onEmailReceived", iGotEmail)

function iGotEmail(newEmail)
{
    if(newEmail.subject.startsWith("URGENT:"))
    {
        newEmail.setPriority(5)
        
        newEmail.reply("Hello,\n Received your email. We will respond on priority basis. \n\nThanks\n")
        return
    }
    else if(newEmail.to.includes("sales@website"))
    {
        newEmail.moveTo("Sales")
        return
    }
}

If you have a Javascript interpreter that allows limited operations running in a sandbox, implementing scriptability will be much easier. That is where a parser for native Go becomes helpful.

Why Javascript?

The scripting language could be any of the many available. Why choose Javascript? Javascript is one of the most popular languages, has detailed documentation, books, StackOverflow answers, has very rich tooling (Translate from ES6, Typescript to ES5). So, you don’t have to teach the language syntax to your users.

Go packages for running Javascript

One option is to use V8 bindings for Go. There are a few packages available.

Looking for pure go implementation takes you to otto and goja

I would go with the native implementation if the requirements of Javascript support does not require full language support. GoJa has Full ECMAScript 5.1 and has unit tests for almost all ES5.1 features. That makes it easy to verify the language features quickly.

Step 1: Running simple Javascript Code from Go

Let’s try getting familiar with the goja library. We run Javascript from Go

package mod2

import (
	"fmt"

	"github.com/dop251/goja"
	"github.com/dop251/goja_nodejs/console"
	"github.com/dop251/goja_nodejs/require"
)

func SimpleJS() {
	vm := goja.New()

	new(require.Registry).Enable(vm)
	console.Enable(vm)

	script := `
		console.log("Hello world - from Javascript inside Go! ")
	`
	fmt.Println("Compiling ... ")
	prog, err := goja.Compile("", script, true)
	if err != nil {
		fmt.Printf("Error compiling the script %v ", err)
		return
	}
	fmt.Println("Running ... \n ")
	_, err = vm.RunProgram(prog)

}

Mapping values from Go, to Javascript and vis-versa

goja has a Value interface defined that is used to convert values to and from Go

type Value interface {
	ToInteger() int64

	ToString() Value
	String() string
	ToFloat() float64
	ToNumber() Value
	ToBoolean() bool
	ToObject(*Runtime) *Object
	SameAs(Value) bool
	Equals(Value) bool
	StrictEquals(Value) bool
	Export() interface{}
	ExportType() reflect.Type
	// contains filtered or unexported methods
}

goja.Runtime has a function ToValue() that will convert from native Go type to internal representation for the Javascript VM. For example, vm.ToValue("a string value") converts from go string.

Similarly, the values returned from the RunTime are also of Value type. You can convert to Go type using the Value interface functions like value.ToBoolean() or value.ToString().

Sometimes, it is more convenient to export to a provided variable. In such cases, use value.ExportTo() You have to provide a pointer of the appropriate type as the second parameter.

Call a Javascript function from Go

Next, let’s try calling a Javascript function from Go


import (
	"fmt"

	"github.com/dop251/goja"
	"github.com/dop251/goja_nodejs/console"
	"github.com/dop251/goja_nodejs/require"
)

func CallJSFunction() {
	vm := goja.New()

	new(require.Registry).Enable(vm)
	console.Enable(vm)

	script := `
		function myFunction(param)
		{
			console.log("myFunction running ...")
			console.log("Param = ", param)
			return "Nice meeting you, Go"
		}
	`

	prog, err := goja.Compile("", script, true)
	if err != nil {
		fmt.Printf("Error compiling the script %v ", err)
		return
	}
	_, err = vm.RunProgram(prog)

	var myJSFunc goja.Callable
	err = vm.ExportTo(vm.Get("myFunction"), &myJSFunc)
	if err != nil {
		fmt.Printf("Error exporting the function %v", err)
		return
	}

	res, err := myJSFunc(goja.Undefined(), vm.ToValue("message from go"))
	if err != nil {
		fmt.Printf("Error calling function %v", err)
		return
	}
	fmt.Printf("Returned value from JS function\n%s \n", res.ToString())

}

goja.Runtime Get() function gets values from the global object. myFunction is defined in the global scope. So you can get the function calling: vm.Get("myFunction"). This returns the Value interface for the function. Then we use the ExportTo function to get the function to a variable of the right type.

The first parameter to the function is the “this” of the Javascript function. Since this function is not meant to use “this” we can pass Undefined(). The second parameter of the Go function is the first parameter of the Javascript function - a simple string. We can convert from Go string to Value interface type by calling RunTime.ToValue() function.

The return value from the function is also of Value type. Since we know that the return value is string, just calling res.ToString() works just fine.

Receiving a Javascript Object back in Go

Making a specialized object available in Javascript

Let’s come back to our original intention of making your App scriptable. For the custom script to be useful, we have to expose parts of the App to the script. For example, in order to write custom script for Google Sheets, you have a SpreadsheetApp object that provides that access. For example, SpreadsheetApp.getActiveSheet() gets the active sheet. Then you can access the rows by calling sheet.getDataRange().getValues().

Now let’s review the sample code in the beginning of this article.

myemail.RegisterHook("onEmailReceived", iGotEmail)

Our scripting environment requires a myemail object through which the user can register a hook that will get called whenever an email is received.

We can create an Object in Go and add it to the globals of the Javascript RunTime.


import (
	"fmt"

	"github.com/dop251/goja"
	"github.com/dop251/goja_nodejs/console"
	"github.com/dop251/goja_nodejs/require"
)

type Hooks struct {
	OnNewEmail         []*goja.Value
	BeforeSendingEmail []func(e *Email)
}

type Email struct {
	Subject string
	Body    string
}

func (h Hooks) Init() {
	h.OnNewEmail = make([]*goja.Value, 0)
	h.BeforeSendingEmail = make([]func(e *Email), 0)
}

func (h *Hooks) TriggerNewEmailEvent(email *Email, vm *goja.Runtime) {

	for _, newEmail := range h.OnNewEmail {
		var newEmailCallBack func(e *Email)
		vm.ExportTo(*newEmail, &newEmailCallBack)
		newEmailCallBack(email)
	}
}

func WorkWithHooks() {
	var hooks Hooks
	hooks.Init()

	vm := goja.New()

	new(require.Registry).Enable(vm)
	console.Enable(vm)

	obj := vm.NewObject()

	obj.Set("RegisterHook", func(hook string, fn goja.Value) {
		hooks.OnNewEmail = append(hooks.OnNewEmail, &fn)
		fmt.Println("Registered the Hook ")
	})

	vm.Set("myemail", obj)

	script := `
		console.log("JS code started ")
		
		myemail.RegisterHook("onEmailReceived", iGotEmail)
		
		function iGotEmail(newEmail)
		{
			console.log("New Email callback received. \n",newEmail.Subject,"\n", newEmail.Body)
		}
	`
	prg, err := goja.Compile("", script, true)
	if err != nil {
		fmt.Printf("Error compiling the script %v ", err)
		return
	}

	_, err = vm.RunProgram(prg)

	email := &Email{
		Subject: "Your order for blue widgets",
		Body:    "Will be delivered in 1.3 micro seconds",
	}

	fmt.Println("Triggering the event ")
	hooks.TriggerNewEmailEvent(email, vm)
}

First we add a method to the custom object that makes it possible for the users to register their “hooks”.

	obj.Set("RegisterHook", func(hook string, fn goja.Value) {
		hooks.OnNewEmail = append(hooks.OnNewEmail, &fn)
		fmt.Println("Registered the Hook ")
	})

Hooks are pointers to functions inside the script. We keep the hooks in a table and call the hooks when an event occurs. Note that we are getting the Value object for the callback from the RunTime and keeping it in the hooks table. In the TriggerNewEmailEvent function, we convert the Value object to function and call it passing the Email object.

func (h *Hooks) TriggerNewEmailEvent(email *Email, vm *goja.Runtime) {

	for _, newEmail := range h.OnNewEmail {
		var newEmailCallBack func(e *Email)
		vm.ExportTo(*newEmail, &newEmailCallBack)
		newEmailCallBack(email)
	}
}

goja can convert from go struct to Javascript object. This is useful in simple cases such as passing value. For more closer integration, we will have to make a custom Object like that of the myemail object.

Exposing more Programmable objects to the script

The sample script at the beginning of this article has one more level of exposure. It calls newEmail.reply() and also newEmail.moveTo() to move the email to another folder. We have to build another goja.Object for the email object.

func makeEmailJSObject(vm *goja.Runtime, email *Email) *goja.Object {
	obj := vm.NewObject()
	obj.Set("subject", email.Subject)
	obj.Set("body", email.Body)
	obj.Set("to", email.To)
	obj.Set("reply", func(body string) {
		fmt.Printf("Replying:\n%s\n", body)
	})
	obj.Set("setPriority", func(p int) {
		fmt.Printf("Set email priority to %d\n", p)
	})

	obj.Set("moveTo", func(folder string) {
		fmt.Printf("Moving email to folder %s\n", folder)
	})

	return obj
}

We can now update our original code to use this function and make use of the new Email object.


import (
	"fmt"

	"github.com/dop251/goja"
	"github.com/dop251/goja_nodejs/console"
	"github.com/dop251/goja_nodejs/require"
)

type Hooks struct {
	OnNewEmail         []*goja.Value
	BeforeSendingEmail []func(e *Email)
}

type Email struct {
	Subject  string
	Body     string
	Priority int
	To       []string
}

func (h Hooks) Init() {
	h.OnNewEmail = make([]*goja.Value, 0)
	h.BeforeSendingEmail = make([]func(e *Email), 0)
}

func (h *Hooks) TriggerNewEmailEvent(email *Email, vm *goja.Runtime) {

	eobj := makeEmailJSObject(vm, email)

	for _, newEmail := range h.OnNewEmail {
		var newEmailCallBack func(*goja.Object)
		vm.ExportTo(*newEmail, &newEmailCallBack)
		newEmailCallBack(eobj)
	}
}

func makeEmailJSObject(vm *goja.Runtime, email *Email) *goja.Object {
	obj := vm.NewObject()
	obj.Set("subject", email.Subject)
	obj.Set("body", email.Body)
	obj.Set("to", email.To)
	obj.Set("reply", func(body string) {
		fmt.Printf("Replying:\n%s\n", body)
	})
	obj.Set("setPriority", func(p int) {
		fmt.Printf("Set email priority to %d\n", p)
	})

	obj.Set("moveTo", func(folder string) {
		fmt.Printf("Moving email to folder %s\n", folder)
	})

	return obj
}

func WorkWithHooks() {
	var hooks Hooks
	hooks.Init()

	vm := goja.New()

	new(require.Registry).Enable(vm)
	console.Enable(vm)

	obj := vm.NewObject()

	obj.Set("RegisterHook", func(hook string, fn goja.Value) {
		switch hook {
		case "onEmailReceived":
			hooks.OnNewEmail = append(hooks.OnNewEmail, &fn)
			fmt.Println("Registered onEmailReceived Hook ")
		}

	})

	vm.Set("myemail", obj)

	script := `
		console.log("JS code started ")
		
		myemail.RegisterHook("onEmailReceived", iGotEmail)

		function iGotEmail(newEmail)
		{
			console.log("newEmail, subject %s ", newEmail.subject,  newEmail.to)
			
			if(newEmail.subject.startsWith("URGENT:"))
			{
				newEmail.setPriority(5)
				
				newEmail.reply("Hello,\n Received your email. We will respond on priority basis. \n\nThanks\n")
				return
			}
			else if(newEmail.to.includes("sales@website"))
			{
				newEmail.moveTo("Sales")
				return
			}
		}
	`
	prg, err := goja.Compile("", script, true)
	if err != nil {
		fmt.Printf("Error compiling the script %v ", err)
		return
	}

	_, err = vm.RunProgram(prg)

	email1 := &Email{
		Subject: "URGENT: Systems down!",
		Body:    "5 of your systems are down at the moment",
		To:      []string{"some@one.cc"},
	}

	fmt.Println("Triggering the urgent email event ")
	hooks.TriggerNewEmailEvent(email1, vm)

	email2 := &Email{
		Subject: "New order received!",
		Body:    "You got new order for 1k blue widgets!",
		To:      []string{"sales@website"},
	}

	fmt.Println("Triggering the sales email event ")
	hooks.TriggerNewEmailEvent(email2, vm)
}

We first trigger the “Urgent” email event and then trigger the “sales” email event. The custom script directs the flow through the two cases handled in the script. That is the beauty of making the system scriptable. Now the user has the power to change and enhance the system behavior which was otherwise not possible.

← Let's build a search query parser in Go Building a simple query parser using PEG in Go →