Nameko for Microservices

Nameko

A microservices framework for Python that lets service developers concentrate on application logic and encourages testability.

Microservices

“The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms“

Example I

                        
    class HelloWorld(object):
        name = “hello”

        @rpc
        def greet(self, friend):
            return “Hello {}!”.format(friend)

Demo!

Nameko Concepts

Entrypoints

To interact with a service

Dependencies

For the service to interact with external things

Example II

                        
    class HelloWorld(object):
        name = “hello”

        cache = CacheClient()  # dependency declaration

        @http("GET", "/hello/<string:friend>")
        def greet(self, request, friend):
            greeting = self.cache.get(friend)

            if greeting is None:
                greeting = “Hello {}!”.format(friend)  # expensive
                self.cache.put(friend, greeting)

            return greeting

                                
    >>> HelloWorld.CacheClient
    ... <CacheClient [unbound] at 0x10cbb2750>    # DependencyProvider
    >>> self.cache
    ... <memcache.Client object at 0x10cbd25d0>   # injected dependency

“... lets service developers concentrate on application logic and encourages testability“

To show this, let's expand our example. Pretend for a minute string formatting is expensive; we want to cache greetings instead of generating each time
This line "cache = CacheClient" declares a dependency from our service to a cache somewhere.
Then in the service method, we can get and put items as needed.

Notice that there's no setup here; the method just uses the cache.
This is possible because the class level attribute is different to the instance level attribute that the method sees when it executes

{{NEXT}} If we inspected them, we'd see that they are different. The class-level attr is an instance of a DependencyProvider. The instance-level attr is actually a memcache.Client object, which is the only part the method needs to use the cache
The DependencyProvider is responsible for returning the actual dependency, which is injected into the service at runtime
All the complexity of the dependency, like managing a connection pool or handing reconnections and retries can be hidden away

One more thing to point out is that I changed the entrypoint: now listens for HTTP requests instead of using RPC.
But note that the application logic hasn't had to change as a result.
In other words: it was a declarative change, had no impact on procedural code in the method

Hopefully you can see how using dependencies lets {{next}} "service developers concentrate on application logic"

Test Helpers

                                
        from mock import call
        from nameko.testing.services import worker_factory

        hello_svc = worker_factory(HelloWorld)

        hello_svc.cache.get.return_value = None

        assert hello_svc.greet("Matt") == "Hello Matt!"
        assert hello_svc.cache.get.call_args_list == [call("Matt")]

“... lets service developers concentrate on application logic and encourages testability“

                                
    import mockcache
    from nameko.testing.services import worker_factory

    fake_cache = mockcache.Client()
    hello_svc = worker_factory(HelloWorld, cache=fake_cache)

    assert hello_svc.greet("Matt") == "Hello Matt!"
    assert fake_cache.get("Matt") == "Hello Matt!"

Nameko also comes with some nice test helpers
We've seen how injecting dependencies keeps service methods clean and simple
It also makes it really easy to isolate them during testing
{{NEXT}}

For example here, using a helper called worker_factory. Useful when unit testing services.
You pass it your service class, and get an instance with deps replaced by mocks. No real memcache cluster reqd.
You can set return values, exercise the methods, verify mocks called appropriately
{{NEXT}}

The worker_factory has another mode of operation where you can provide an alternative dependency.
In this case Mockcache; much nicer interface than a plain mock for this situation; (no need to set up return value)

There are similar helpers for non-unit tests, which let you run services with mocked dependencies, and other goodies too.

Extensible

"Built-in" Extensions

AMQP RPC
AMQP Pub-Sub
HTTP based on Werkzeug
Timer
Websockets

"Community" Extensions

nameko-sentry
nameko-sqlalchemy
nameko-redis
nameko-slack
... and more!

Final feature I want to talk about is extensibility
Architecture of using entrypoints for reqs "in" and dependencies for reqs "out", is similar to "ports-and-adapters" and "hexagonal arch" patterns.
... where your application logic sits between pluggable inputs and outputs

Actually all Entrypoints and DependencyProviders are implemented as so-called "extensions"

And it's relatively easily to create new ones.

The idea is that you can make whatever technology choices you want, but we do include some so it's useful out of the box

The ones included with framework are called "built-in" extensions {{NEXT}}
The rpc decorator we've seen is AMQP based RPC implementation for Req-Resp over message bus
There is also publish-subscribe impl for async messaging over AMQP
HTTP, based on Werkzeug
And timer for cron-like periodic actions, and websockets

And there are a growing collection of "community" extensions...
Including nameko-slack so you can write chat-bots in nameko

Nameko

Made for microservices
Enables you to write clean, highly testable code
Designed to be extended
Useful out of the box

Docs: https://nameko.readthedocs.org

Repo: https://github.com/nameko/nameko

Help: https://groups.google.com/forum/#!forum/nameko-dev

Thank you

Slides: https://mattbennett.github.io/nameko-london-python-presentation

Demo Files: https://github.com/mattbennett/nameko-london-python-demo