What's new in RESTHeart 1.1.0

 

What's new in RESTHeart 1.1.0

08 December 2015  • SoftInstigate Team
Categories: en, technology  • Tags: restheart

We are proud to announce the release of RESTHeart v 1.1.0. This is a big update, with 1 new feature, 7 improvements and 5 bugfixes. We got several feedbacks from the community that allowed us to focus on real user needs: you asked, we listened!

  • Aggregation Operations
  • Compact HAL representation
  • Define the connection to MongoDB using Mongo URI
  • Allow the connect to MongoDB with only read or readWrite role

For more information refer to the release notes.

Aggregation Operations

RESTHeart 1.1.0 supports both aggregation pipeline and mapReduce functions.

“Aggregations operations process data records and return computed results. Aggregation operations group values from multiple documents together, and can perform a variety of operations on the grouped data to return a single result.”

Let’s see it in action with a simple example: the collection invoices has the properties amount and customer. We are going to create an aggregation pipeline operation that computes the total invoices amount per customer.

In order to achieve this, we create the collection invoices with the aggrs metadata defining the aggregation pipeline operation total-by-cust bound at URI /db/invoices/_aggrs/total-by-cust

$ http PUT 127.0.0.1:8080/db/collection aggrs:='[\
{"type":"pipeline",\
"uri":"total-by-cust",\
"stages": [\
{"_$match": { "customer": { "_$var": "customer"}}},\
{"_$group": { "_id": "$customer", "total": {"_$sum": "$total"}\
}}]}]'
HTTP/1.1 201 Created
...

Note the _$var operator used in the match stage. This is a RESTHeart operator that allows to pass the variable customer to the aggregation via the avars query parameter.

Let’s create few invoice documents:

$ http POST 127.0.0.1:8080/test/collection total:=1000 customer=C1
HTTP/1.1 201 Created
...

$ http POST 127.0.0.1:8080/test/collection total:=2000 customer=C1
HTTP/1.1 201 Created
...

$ http POST 127.0.0.1:8080/test/collection total:=3000 customer=C1
HTTP/1.1 201 Created
...

$ http POST 127.0.0.1:8080/test/collection total:=1000 customer=C2
HTTP/1.1 201 Created
...

Let’s now get the collection and see the aggregation operation endpoint between its _links

http GET 127.0.0.1:8080/test/collection
HTTP/1.1 200 OK

{
    "_collection-props-cached": false,
    "_embedded": { ... },
    "_etag": {
        "$oid": "56669d1cdf004bf209a19171"
    },
    "_id": "collection",
    "_links": {
        "curies": [],
        "self": {
            "href": "/test/collection"
        },
        "total-by-cust": {
            "href": "/test/collection/_aggrs/total-by-cust"
        }
    },
    "_returned": 4,
    "aggrs": [ ... ]
}

We now call the aggregation pipeline operation, passing the variable customer as “C1”

http GET 127.0.0.1:8080/test/collection/_aggrs/total-by-cust?avars='{"customer":"C1"}'
HTTP/1.1 200 OK
...

{
    "_embedded": {
        "rh:result": [
            {
                "_id": "C1",
                "total": 6000
            }
        ]
    },
    "_links": {
        "curies": [],
        "self": {
            "href": "/test/collection/_aggrs/total-by-cust?avars={\"customer\":\"C1\"}"
        }
    },
    "_returned": 1,
    "_size": 1,
    "_total_pages": 1
}

More information on Aggregation documentation section.

Compact HAL representation

We got several feedbacks about the HAL representation format being too much verbose. HAL is a great choice in our opinion: it is simple, elegant, well defined and, most important, it is an established format. It is also very flexible, allowing to tailor it to your needs.

One interesting feature of HAL is the ability to attach the API documentation to representation of the resource. This is why RESTHeart versions 1.0.x and before used to add curies and link templates in the response. Some useful properties and links used to be added as well, for instance _type (the resource type such as DB, COLLECTION or DOCUMENT) and pagination links.

We got feedbacks about this additional data being actually an overhead, despite it is useful at development time.

RESTHeart v1.1.0 still uses HAl but it avoids to add curies, link templates, pagination likns and special properties by default. On the other hand, adding the hal=f (f for full) query parameter brings back them all.

Example

In this example, the new HAL format is 71% smaller.

Collection representation with HAL compact mode (default)

$http GET 127.0.0.1:8080/test/bands

HTTP/1.1 200 OK
....

{
    "_collection-props-cached": false,
    "_created_on": "2015-09-10T10:47:14Z",
    "_embedded": {
        "rh:doc": [
            {
                "_etag": {
                    "$oid": "55f16029c2e65448b566d191"
                },
                "_id": "The Cure",
                "_links": {
                    "self": {
                        "href": "/test/bands/The Cure"
                    }
                },
                "albums": [
                    "Disintegration",
                    "Three Imaginary Boys",
                    "Seventeen Seconds"
                ]
            },
            {
                "_etag": {
                    "$oid": "562e962ec2e6a475e8fe3afe"
                },
                "_id": "Pink Floyd",
                "_links": {
                    "self": {
                        "href": "/test/bands/Pink Floyd"
                    }
                },
                "albums": [
                    "The Piper at the Gates of Dawn",
                    "A Saucerful of Secrets",
                    "More",
                    "Ummagumma",
                    "Atom Heart Mother",
                    "Meddle",
                    "Obscured by Clouds",
                    "The Dark Side of the Moon",
                    "Wish You Were Here",
                    "Animals",
                    "The Wall",
                    "The Final Cut",
                    "A Momentary Lapse of Reason",
                    "The Division Bell",
                    "The Endless River"
                ]
            }
        ]
    },
    "_etag": {
        "$oid": "55f15fb2c2e65448b566d18c"
    },
    "_id": "bands",
    "_links": {
        "curies": [],
        "self": {
            "href": "/test/bands"
        }
    },
    "_returned": 2,
    "descr": "music bands",
}

Collection representation with HAL full mode (with hal=c qparam)

$ http GET 127.0.0.1:8080/test/bands?hal=f
HTTP/1.1 200 OK
...

{
    "_collection-props-cached": false,
    "_created_on": "2015-09-10T10:47:14Z",
    "_embedded": {
        "rh:doc": [
            {
                "_etag": {
                    "$oid": "55f16029c2e65448b566d191"
                },
                "_id": "The Cure",
                "_lastupdated_on": "2015-09-10T10:49:13Z",
                "_links": {
                    "self": {
                        "href": "/test/bands/The Cure"
                    }
                },
                "_type": "DOCUMENT",
                "albums": [
                    "Disintegration",
                    "Three Imaginary Boys",
                    "Seventeen Seconds"
                ]
            },
            {
                "_etag": {
                    "$oid": "562e962ec2e6a475e8fe3afe"
                },
                "_id": "Pink Floyd",
                "_lastupdated_on": "2015-10-26T21:07:58Z",
                "_links": {
                    "self": {
                        "href": "/test/bands/Pink Floyd"
                    }
                },
                "_type": "DOCUMENT",
                "albums": [
                    "The Piper at the Gates of Dawn",
                    "A Saucerful of Secrets",
                    "More",
                    "Ummagumma",
                    "Atom Heart Mother",
                    "Meddle",
                    "Obscured by Clouds",
                    "The Dark Side of the Moon",
                    "Wish You Were Here",
                    "Animals",
                    "The Wall",
                    "The Final Cut",
                    "A Momentary Lapse of Reason",
                    "The Division Bell",
                    "The Endless River"
                ]
            }
        ]
    },
    "_etag": {
        "$oid": "55f15fb2c2e65448b566d18c"
    },
    "_id": "bands",
    "_lastupdated_on": "2015-09-10T10:47:14Z",
    "_links": {
        "curies": [
            {
                "href": "http://restheart.org/curies/1.0/{rel}.html",
                "name": "rh",
                "templated": true
            }
        ],
        "first": {
            "href": "/test/bands?pagesize=100&hal=f"
        },
        "next": {
            "href": "/test/bands?page=2&pagesize=100&hal=f"
        },
        "rh:coll": {
            "href": "/test/{collname}",
            "templated": true
        },
        "rh:db": {
            "href": "/test"
        },
        "rh:document": {
            "href": "/test/bands/{docid}{?id_type}",
            "templated": true
        },
        "rh:filter": {
            "href": "/test/bands{?filter}",
            "templated": true
        },
        "rh:indexes": {
            "href": "/test/bands/_indexes"
        },
        "rh:paging": {
            "href": "/test/bands{?page}{&pagesize}",
            "templated": true
        },
        "rh:sort": {
            "href": "/test/bands{?sort_by}",
            "templated": true
        },
        "self": {
            "href": "/test/bands?hal=f"
        }
    },
    "_returned": 2,
    "_type": "COLLECTION",
    "descr": "music bands",
}

Define the connection to MongoDB using Mongo URI

In previous versions of RESTHeart, the connection with MongoDB used to be specified with mongo-server and mongo-credentials configuration options.

Than we got a question on github where merrickkw suggested us to use Mongo URI instead.

And we listened!

The advantages are impressive since the MongoDB connection string URI allows to define several options (such as write concern, read options),supports SSL/TLS and all authentication mechanisms (including Kerberos and LDAP).

Simple change, great improvement!

Allow the connect to MongoDB with only read or readWrite role

In previous versions of RESTHeart, the connection with MongoDB required the database user to have dbAdmin role and listDataBases permission.

This had several issues, from security (it’s always a good practise to have the database user with as less privileges as possible) to problems with mongodb hosting services (such as compose.io or mongo lab) that usually provides just readWrite role.

The problem relied in the need to check the existence of the database and collection before allowing any GET operation. This because the HTTP protocol mandates that GET operation are always safe, i.e. they don’t change the status of the server. On the other side, MongoDB creates the resource on read if it does not exist before.

That’s why we needed to execute this check before actually query the database and this required the listDatabases permission and the dbAdmin role. In RESTHeart 1.1.0 we found a way to accomplish this that only requires the read role.

Now you can use the mongodb user with just read (for db read only access) or readWrite role (for read+write access).

Curious how? See the magic commits c98f8f8 and 003e3e6


 

Comments