请选择 进入手机版 | 继续访问电脑版

技术控

    今日:4| 主题:61818
收藏本版 (1)
最新软件应用技术尽在掌握

[其他] Microservices in Flask

[复制链接]
披着蚊帐当婚纱 发表于 2016-10-10 18:32:56
254 0
I recently gave a talk about microservices in Flask on Wrocpy . This blog post is a translation of that talk into English.
  Table of Contents:
      Monolithic application

   Microservices talk usually starts with a monolithic application. In my case, it is the same. I work on a project where I have a big monolithic application. If I wanted to take some part of it and make some microservice from for instance django app it would be impossible. There are too many internal imports from every part of application:
  
  1. from app.users.models import UserSiteProfile
  2. from app.utils.cache import cache_key_user
  3. from app.sites.models import SiteProfile
  4. from app.sites.utils import site_terms
  5. from app.utils.users import get_homepage_url
  6. from app.utils.views import ThemedFormView, ThemedTemplateView
  7. from app.authentication import signals
  8. from app.authentication.forms import AuthForm, EmailForm
  9. from app.authentication.idp import (
  10.     is_valid_idp, MultipleIDPError, saml_available,
  11.     site_idp, user_idp_lookup)
  12. from app.authentication.loginchecks import (
  13.     check_account_expiration, get_site_login_errors,
  14.     pre_login_checks
  15. )
  16. from app.saml.utils import site_specific_config_loader
  17. from app.saml.views import _get_subject_id
复制代码
  Snippet from above presents exemplary imports of random python module in my project.
    So where are these microservices? In my team, we decided to have new parts of the application made as a microservices. So right now from the architectural point of view I have a big monolithic application and small microservices that are around this big thing, like in this picture:
   

Microservices in Flask

Microservices in Flask-1-技术控-recently,English,usually,import,wanted
    Picture from link .
        We choose the flask as a tool that will be used in our microservices. It doesn't set any boundaries and it's flexible but don't have batteries included . Let's start with the first flask extension that helps us building microservices:
        Django has a set of powerful commands available. To enable such a feature in flask you needFlask-Script. It allows you to create commands such as runserver or shell . In listing below I created a runserver command:
   
  1. from flask.ext.script import Manager, Shell, Server
  2. from my_app.application import app
  3. manager = Manager(app)
  4. manager.add_command(
  5.     'runserver',
  6.     Server(host='0.0.0.0', port=7000, use_debugger=True)
  7. )
复制代码
          To communicate between microservices I use REST. To ease yourself when creating resources and endpoints you can useFlask-RESTful. It is superuseful when you need to create REST API. It is easy- you specify endpoint with resource and rest is done by Flask-RESTful. It also has request parsing and it is very easy to create other representations like xml. The snippet below shows it:
   
  1. from flask_restful import Resource
  2. class MyResource(Resource):
  3.     def get():
  4.         return {'OK'}
  5. @api.representation('application/xml')
  6. def output_xml(data, code, headers=None):
  7.     response = make_response(dicttoxml.dicttoxml(data), code)
  8.     inject_headers(response, headers)
  9.     return response
复制代码
          Marshallow & flask-marshallow

     To serialize or deserialize objects you can use flask-marshallow . In code below, I created a simple Schema with id, name and url. Then when the user enters /api/custom/1 I can easily serialize objects from a database and return JSON.
   
  1. ma = Marshmallow(app)
  2. class CustomSchema(ma.Schema):
  3.     id = ma.Int(dump_only=True)
  4.     name = ma.Str(required=True)
  5.     url = ma.Url(relative=True)
  6.     @app.route('/api/custom/<id>')
  7.     def users():
  8.         all_obj = Object.all()
  9.         result = object_schema.dump(all_obj)
  10.         return jsonify(result.data)
复制代码
          Most of the modern frameworks have support for ORM- the same is with Flask. There is an extension called Flask-SQLAchemy that adds support for SQLAlchemy . Take this snippet for instance:
   
  1. from flask.ext.sqlalchemy import SQLAlchemy
  2. db = SQLAlchemy(app)
  3. class MyModel(db.Model):
  4.     GROUP = 'group'
  5.     USER = 'user'
  6.     TYPES = [
  7.         (GROUP, GROUP),
  8.         (USER, USER),
  9.     ]
  10.     __tablename__ = 'my_model'
  11.     id = db.Column(db.Integer, primary_key=True)
  12.     model_type = db.Column(ChoiceType(TYPES))
  13.     model_value_id = db.Column(db.Integer, db.ForeignKey('model_value.id'))
  14.     value = db.Column(db.String(1024))
  15.     def __init__(self, model_type, model_value_id, value):
  16.         self.share_type = share_type
  17.         self.rule_value_id = rule_value_id
  18.         self.value = value
复制代码
    I created MyModel class that will be translated to the table in a database. I also add columns to that table like model_type , model_value_id or value .
    Flask SQLAlchemy is layer sitting on top of SQLAlchemy so you can use all advantages of ORM like having queries written in python.
           When your database keeps getting larger there is a need for database migrations to make sure that everyone has the same database structure. To accomplish that we useFlask-Migrate. It is using Alembic under the hood so you have to make sure that adjust your migration files after generation. Example migration file can look as follows:
   
  1. def upgrade():
  2.     op.create_table(
  3.         'my_model',
  4.         sa.Column('id', sa.Integer(), nullable=False),
  5.         sa.Column(
  6.             'model_type',
  7.             sqlalchemy_utils.types.choice.ChoiceType(TYPES),
  8.             nullable=True
  9.         ),
  10.         sa.PrimaryKeyConstraint('id'),
  11.         sa.Column('value', sa.String(length=1024), nullable=True),
  12.     )
  13.     op.create_table(
  14.         'my_model_values',
  15.         sa.Column('id', sa.Integer(), nullable=False),
  16.         sa.Column('model_value_id', sa.Integer(), nullable=True),
  17.         sa.Column('value', sa.String(length=1024), nullable=True),
  18.         sa.ForeignKeyConstraint(['model_value_id'], ['my_model.id']),
  19.         sa.PrimaryKeyConstraint('id')
  20.     )
复制代码
    In snipped above I created two tables: my_model and my_model_values with respective columns. Also my_model_values has ForeignKey relation to my_model by their ids.
       During the development of microservices, we write unit tests as well as integration ones. Testing how well microservices behave with each other can be tricky: we mock whole external services. Because of that, we need to keep them up to date with real microservices. Nature of this system causes some difficulties while an error occurs: I got an error from external microservice in most cases with a form of HTTP status code and a small message in JSON or XML. Then I need to debug not only my microservice but also external one.
     After testing is done we deploy microservice using few tools:
    We use puppet for managing and provisioning our microservices. Especially we use an R10k module for puppet: gtihub link .
        To make sure that every microservice has the same structure we also use cookiecutter . Thanks to that puppet knows that config file is always in this location or there will be logs stored there. Example microservice structure will look as follows:
   
  1. └── flask_microservice
  2.     ├── AUTHORS.rst
  3.     ├── debian
  4.     ├── docs
  5.     │   ├── make.bat
  6.     │   ├── Makefile
  7.     │   └── source
  8.     │       ├── authors.rst
  9.     │       ├── conf.py
  10.     │       ├── contributing.rst
  11.     │       ├── history.rst
  12.     │       ├── index.rst
  13.     │       ├── readme.rst
  14.     │       ├── technical_details.rst
  15.     │       └── usage.rst
  16.     |── HISTORY.rst
  17.     ├── MANIFEST.in
  18.     ├── README.rst
  19.         ├── requirements.txt
  20.         ├── setup.cfg
  21.         ├── setup.py
  22.         ├── src
  23.         │   ├── flask_microservice
  24.         │   │   ├── application.py
  25.         │   │   ├── default_config.ini
  26.         │   │   ├── __init__.py
  27.         │   │   └── manage.py
  28.         │   └── tests
  29.         │       ├── conftest.py
  30.         │       └── test_flask_microservice.py
  31.         └── tox.ini
复制代码
      To distribute packages we usedh-virtualenv. This basically is taking python virtual enviroments and packing it to deb packages. So to have new code released we just run jenkins job to create new deb.
    That's all for today! The slides from this presentation can be found here: presentation . Do you also use flask to build microservices? Please give your comments about that.
   Special thanks to Kasia for being editor for this post. Thank you.
    Cover image by NASA JPL - Nasa JPL under CC0 Public Domain .
我要投稿

推荐阅读


回页顶回复上一篇下一篇回列表
手机版/c.CoLaBug.com ( 粤ICP备05003221号 | 文网文[2010]257号 | 粤公网安备 44010402000842号 )

© 2001-2017 Comsenz Inc.

返回顶部 返回列表