My vision on Data Analysis is that there is continuum between explanatory models on one side and predictive models on the other side. The decisions you make during the modeling process depend on your goal. Let’s take Customer Churn as an example, you can ask yourself why are customers leaving? Or you can ask yourself which customers are leaving? The first question has as its primary goal to explain churn, while the second question has as its primary goal to predict churn. These are two fundamentally different questions and this has implications for the decisions you take along the way. The predictive side of Data Analysis is closely related to terms like Data Mining and Machine Learning.
SPSS & SAS
When we’re looking at SPSS and SAS, both of these languages originate from the explanatory side of Data Analysis. They are developed in an academic environment, where hypotheses testing plays a major role. This makes that they have significant less methods and techniques in comparison to R and Python. Nowadays, SAS and SPSS both have data mining tools (SAS Enterprise Miner and SPSS Modeler), however these are different tools and you’ll need extra licenses.
I have spent some time to build extensive macros in SAS EG to seamlessly create predictive models, which also does a decent job at explaining the feature importance. While a Neural Network may do a fair job at making predictions, it is extremely difficult to explain such models, let alone feature importance. The macros that I have built in SAS EG does precisely the job of explaining the features, apart from producing excellent predictions.
Open source TOOLS: R & PYTHON
One of the major advantages of open source tools is that the community continuously improves and increases functionality. R was created by academics, who wanted their algorithms to spread as easily as possible. R has the widest range of algorithms, which makes R strong on the explanatory side and on the predictive side of Data Analysis.
Python is developed with a strong focus on (business) applications, not from an academic or statistical standpoint. This makes Python very powerful when algorithms are directly used in applications. Hence, we see that the statistical capabilities are primarily focused on the predictive side. Python is mostly used in Data Mining or Machine Learning applications where a data analyst doesn’t need to intervene. Python is therefore also strong in analyzing images and videos. Python is also the easiest language to use when using Big Data Frameworks like Spark. With the plethora of packages and ever improving functionality, Python is a very accessible tool for data scientists.
MACHINE LEARNING MODELS
While procedures like Logistic Regression are very good at explaining the features used in a prediction, some others like Neural Networks are not. The latter procedures may be preferred over the former when it comes to only prediction accuracy and not explaining the models. Interpreting or explaining the model becomes an issue for Neural Networks. You can’t just peek inside a deep neural network to figure out how it works. A network’s reasoning is embedded in the behavior of numerous simulated neurons, arranged into dozens or even hundreds of interconnected layers. In most cases the Product Marketing Officer may be interested in knowing what are the factors that are most important for a specific advertising project. What can they concentrate on to get the response rates higher, rather than, what will be their response rate, or revenues in the upcoming year. These questions are better answered by procedures which can be interpreted in an easier way. This is a great article about the technical and ethical consequences of the lack of explanations provided by complex AI models.
Procedures like Decision Trees are very good at explaining and visualizing what exactly are the decision points (features and their metrics). However, those do not produce the best models. Random Forests, Boosting are the procedures which use Decision Trees as the basic starting point to build the predictive models, which are by far some of the best methods to build sophisticated prediction models.
While Random Forests use fully grown (highly complex) Trees, and by taking random samples from the training set (a process called Bootstrapping), then each split uses only a proper subset of features from the entire feature set to actually make the split, rather than using all of the features. This process of bootstrapping helps with lower number of training data (in many cases there is no choice to get more data). The (proper) subsetting of the features has a tremendous effect on de-correlating the Trees grown in the Forest (hence randomizing it), leading to a drop in Test Set error. A fresh subset of features is chosen at each step of splitting, making the method robust. The strategy also stops the strongest feature from appearing each time a split is considered, making all the trees in the forest similar. The final result is obtained by averaging the result over all trees (in case of Regression problems), or by taking a majority class vote (in case of classification problem).
On the other hand, Boosting is a method where a Forest is grown using Trees which are NOT fully grown, or in other words, with Weak Learners. One has to specify the number of trees to be grown, and the initial weights of those trees for taking a majority vote for class selection. The default weight, if not specified is the average of the number of trees requested. At each iteration, the method fits these weak learners, finds the residuals. Then the weights of those trees which failed to predict the correct class is increased so that those trees can concentrate better on the failed examples. This way, the method proceeds by improving the accuracy of the Boosted Trees, stopping when the improvement is below a threshold. One particularly implementation of Boosting, AdaBoost has very good accuracy over other implementations. AdaBoost uses Trees of depth 1, known as Decision Stump as each member of the Forest. These are slightly better than random guessing to start with, but over time they learn the pattern and perform extremely well on test set. This method is more like a feedback control mechanism (where the system learns from the errors). To address overfitting, one can use the hyper-parameter Learning Rate (lambda) by choosing values in the range: (0,1]. Very small values of lambda will take more time to converge, however larger values may have difficulty converging. This can be achieved by a iterative process to select the correct value for lambda, plotting the test error rate against values of lambda. The value of lambda with the lowest test error should be chosen.
In all these methods, as we move from Logistic Regression, to Decision Trees to Random Forests and Boosting, the complexity of the models increase, making it almost impossible to EXPLAIN the Boosting model to marketers/product managers. Decision Trees are easy to visualize, Logisitic Regression results can be used to demonstrate the most important factors in a customer acquisition model and hence will be well received by business leaders. On the other hand, the Random Forest and Boosting methods are extremely good predictors, without much scope for explaining. But there is hope: These models have functions for revealing the most important variables, although it is not possible to visualize why.
USING A BALANCED APPROACH
So I use a mixed strategy: Use the previous methods as a step in Exploratory Data Analysis, present the importance of features, characteristics of the data to the business leaders in phase one, and then use the more complicated models to build the prediction models for deployment, after building competing models. That way, one not only gets to understand what is happening and why, but also gets the best predictive power. In most cases that I have worked, I have rarely seen a mismatch between the explanation and the predictions using different methods. After all, this is all math and the way of delivery should not change end results. Now that’s a happy ending for all sides of the business!
Why Do We Use External Persistent Storage for Redis Mesos Containers?
Since Redis is an in-memory database, an instance/service restart will result in loss of data. To counter this, it is always advisable to snapshot the Redis in-memory database from time to time.
This helps Redis instance to recover from the point in time failure.
In DCOS, Redis is deployed as a stateless service. To make it a stateful and persistent data, we can configure local volumes or external volumes.
The disadvantage of having a local volume mapped to Mesos containers is when a slave node goes down, your local volume becomes unavailable, and the data loss occurs.
However, with external persistent volumes, as they are available on each node of the DCOS cluster, a slave node failure does not impact the data availability.
Rex-Ray
REX-Ray is an open source, storage management solution designed to support container runtimes such as Docker and Mesos.
REX-Ray enables stateful applications such as databases to persist and maintain its data after the life cycle of the container has ended. Built-in high availability enables orchestrators such as Docker Swarm, Kubernetes, and Mesos Frameworks like Marathon to automatically orchestrate storage tasks between hosts in a cluster.
Built on top of the libStorage framework, REX-Ray’s simplified architecture consists of a single binary and runs as a stateless service on every host using a configuration file to orchestrate multiple storage platforms.
Objective: To create a Redis service in DC/OS environment with persistent storage.
Warning: The Persistent Volume feature is still in beta Phase for DC/OS Version 1.11.
Prerequisites:
Make sure the rexray service is running and is in a healthy state for the cluster.
Steps:
Click on the Add button in Services component of DC/OS GUI.
Click on JSON Configuration.
Note: For persistent storage, below code should be added in the normal Redis service configuration JSON file to mount external persistent volumes.
Make sure the service is up and in a running state:
If you look closely, the service was suspended and respawned on a different slave node. We populated the database with dummy data and saved the snapshot in the data directory.
When the service did come upon a different node 10.0.3.204, the data persisted and the volume was visible on the new node.
To connect with Redis service, use below host:port in your applications:
redis.marathon.l4lb.thisdcos.directory:6379
Conclusion
We learned about Standalone Redis Service deployment from DCOS catalog on DCOS. Also, we saw how to add Persistent storage to it using RexRay. We also learned how RexRay automatically manages volumes over AWS ebs and how to integrate them in DCOS apps/services. Finally, we saw how other applications can communicate with this Redis service.
It is amazing how the software industry has evolved. Back in the day, a software was a simple program. Some of the first software applications like The Apollo Missions Landing modules and Manchester Baby were basic stored procedures. Software was primarily used for research and mathematical purposes.
The invention of personal computers and the prominence of the Internet changed the software world. Desktop applications like word processors, spreadsheets, and games grew. Websites gradually emerged. Back then, simple pages were delivered to the client as static documents for viewing. By the mid-1990s, with Netscape introducing client-side scripting language, JavaScript and Macromedia bringing in Flash, the browser became more powerful, allowing websites to become richer and more interactive. In 1999, the Java language introduced Servlets. And thus born the Web Application. Nevertheless, these developments and applications were still simpler. Engineers didn’t emphasize enough on structuring them and mostly built unstructured monolithic applications.
The advent of disruptive technologies like cloud computing and Big data paved the way for more intricate, convolute web and native mobile applications. From e-commerce and video streaming apps to social media and photo editing, we had applications doing some of the most complicated data processing and storage tasks. The traditional monolithic way now posed several challenges in terms of scalability, team collaboration and integration/deployment, and often led to huge and messy The Ball Of Mud codebases.
To untangle this ball of software, came in a number of service-oriented architectures. The most promising of them was Microservices – breaking an application into smaller chunks that can be developed, deployed and tested independently but worked as a single cohesive unit. Its benefits of scalability and ease of deployment by multiple teams proved as a panacea to most of the architectural problems. A few front-end architectures also came up, such as MVC, MVVM, Web Components, to name a few. But none of them were fully able to reap the benefits of Microservices.
Micro Frontends first came up in ThoughtWorks Technology Radar where they assessed, tried and eventually adopted the technology after noticing significant benefits. It is a Microservice approach to front-end web development where independently deliverable front-end applications are composed as a whole.
With Microservices, Micro Frontends breaks the last monolith to create a complete Micro-Architecture design pattern for web applications. It is entirely composed of loosely coupled vertical slices of business functionality rather than in horizontals. We can term these verticals as ‘Microapps’. This concept is not new and has appeared in Scaling with Microservices and Vertical Decomposition. It first presented the idea of every vertical being responsible for a single business domain and having its presentation layer, persistence layer, and a separate database. From the development perspective, every vertical is implemented by exactly one team and no code is shared among different systems.
Fig: Micro Frontends with Microservices (Micro-architecture)
Why Micro Frontends?
A microservice architecture has a whole slew of advantages when compared to monolithic architectures.
Ease of Upgrades – Micro Frontends build strict bounded contexts in the application. Applications can be updated in a more incremental and isolated fashion without worrying about the risks of breaking up another part of the application.
Scalability – Horizontal scaling is easy for Micro Frontends. Each Micro Frontend has to be stateless for easier scalability.
Ease of deployability: Each Micro Frontend has its CI/CD pipeline, that builds, tests and deploys it to production. So it doesn’t matter if another team is working on a feature and has pushed a bug fix or if a cutover or refactoring is taking place. There should be no risks involved in pushing changes done on a Micro Frontend as long as there is only one team working on it.
Team Collaboration and Ownership: The Scrum Guide says that “Optimal Development Team size is small enough to remain nimble and large enough to complete significant work within a Sprint”. Micro Frontends are perfect for multiple cross-functional teams that can completely own a stack (Micro Frontend) of an application from UX to Database design. In case of an E-commerce site, the Product team and the Payment team can concurrently work on the app without stepping on each other’s toes.
Micro Frontend Integration Approaches
There is a multitude of ways to implement Micro Frontends. It is recommended that any approach for this should take a Runtime integration route instead of a Build Time integration, as the former has to re-compile and release on every single Micro Frontend to release any one of the Micro Frontend’s changes.
We shall learn some of the prominent approaches of Micro Frontends by building a simple Pet Store E-Commerce site. The site has the following aspects (or Microapps, if you will) – Home or Search, Cart, Checkout, Product, and Contact Us. We shall only be working on the Front-end aspect of the site. You can assume that each Microapp has a microservice dedicated to it in the backend. You can view the project demo here and the code repository here. Each way of integration has a branch in the repo code that you can check out to view.
Single Page Frontends –
The simplest way (but not the most elegant) to implement Micro Frontends is to treat each Micro Frontend as a single page.
Fig: Single Page Micro Frontends: Each HTML file is a frontend.
!DOCTYPE html><htmllang="zxx"><head> <title>The MicroFrontend - eCommerce Template</title></head><body> <headerclass="header-section header-normal"> <!-- Header is repeated in each frontend which is difficult to maintain --> .... .... </header <main> </main> <footer<!--Footerisrepeatedineachfrontendwhichmeanswehavetomultiplechangesacrossallfrontends--> </footer> <script> <!-- Cross Cutting features like notification, authentication are all replicated in all frontends--> </script></body>
It is one of the purest ways of doing Micro Frontends because no container or stitching element binds the front ends together into an application. Each Micro Frontend is a standalone app with each dependency encapsulated in it and no coupling with the others. The flipside of this approach is that each frontend has a lot of duplication in terms of cross-cutting concerns like headers and footers, which adds redundancy and maintenance burden.
JavaScript Rendering Components (Or Web Components, Custom Element)-
As we saw above, single-page Micro Frontend architecture has its share of drawbacks. To overcome these, we should opt for an architecture that has a container element that builds the context of the app and the cross-cutting concerns like authentication, and stitches all the Micro Frontends together to create a cohesive application.
// A virtual class from which all micro-frontends would extendclassMicroFrontend {beforeMount() {// do things before the micro front-end mounts }onChange() {// do things when the attributes of a micro front-end changes }render() {// html of the micro frontendreturn'<div></div>'; }onDismount() {// do things after the micro front-end dismounts }}
classCartextendsMicroFrontend {beforeMount() {// get previously saved cart from backend }render() {return`<!-- Page --> <div class="page-area cart-page spad"> <div class="container"> <div class="cart-table"> <table> <thead> ..... ` }addItemToCart(){... }deleteItemFromCart () {... }applyCouponToCart() {... }onDismount() {// save Cart for the user to get back to afterwards }}
<!DOCTYPE html><htmllang="zxx"><head> <title>PetStore - because Pets love pampering</title> <metacharset="UTF-8 <link rel="stylesheet"href="css/style.css"/></head><body> <!-- Header section --> <headerclass="header-section"> .... </header> <!-- Header section end --> <mainid='microfrontend'> <!-- This is where the Micro-frontend gets rendered by utility renderMicroFrontend.js--> </main> <!-- Header section --> <footerclass="header-section"> .... </footer> <!-- Footer section end --> <scriptsrc="frontends/MicroFrontend.js"></script> <scriptsrc="frontends/Home.js"></script> <scriptsrc="frontends/Cart.js"></script> <scriptsrc="frontends/Checkout.js"></script> <scriptsrc="frontends/Product.js"></script> <scriptsrc="frontends/Contact.js"></script> <scriptsrc="routes.js"></script> <scriptsrc="renderMicroFrontend.js"></script>
functionrenderMicroFrontend(pathname) {constmicroFrontend= routes[pathname || window.location.hash];constroot= document.getElementById('microfrontend'); root.innerHTML = microFrontend ?newmicroFrontend().render():newHome().render();$(window).scrollTop(0);}$(window).bind( 'hashchange', function(e) { renderFrontend(window.location.hash); });renderFrontend(window.location.hash);utility routes.js (A map of the hash route to the Microfrontend class)constroutes= {'#': Home,'': Home,'#home': Home,'#cart': Cart,'#checkout': Checkout,'#product': Product,'#contact': Contact,};
As you can see, this approach is pretty neat and encapsulates a separate class for Micro Frontends. All other Micro Frontends extend from this. Notice how all the functionality related to Microapp is encapsulated in the respective Micro Frontend. This makes sure that concurrent work on a Micro Frontend doesn’t mess up some other Micro Frontends.
Everything will work in a similar paradigm when it comes to Web Components and Custom Elements.
React
With the client-side JavaScript frameworks being very popular, it is impossible to leave React from any Front End discussion. React being a component-based JS library, much of the things discussed above will also apply to React. I am going to discuss some of the technicalities and challenges when it comes to Micro Frontends with React.
Styling
Since there should be minimum sharing of code between any Micro Frontend, styling the React components can be challenging, considering the global and cascading nature of CSS. We should make sure styles are targeted on a specific Micro Frontend without spilling over to other Micro Frontends. Inline CSS, CSS in JS libraries like Radium, and CSS Modules, can be used with React.
Redux
Using React with Redux is kind of a norm in today’s front-end world. The convention is to use Redux as a single global store for the entire app for cross application communication. A Micro Frontend should be self-contained with no dependencies. Hence each Micro Frontend should have its own Redux store, moving towards a multiple Redux store architecture.
Other Noteworthy Integration Approaches –
Server-side Rendering – One can use a server to assemble Micro Frontend templates before dispatching it to the browser. SSI techniques can be used too.
iframes – Each Micro Frontend can be an iframe. They also offer a good degree of isolation in terms of styling, and global variables don’t interfere with each other.
Summary
With Microservices, Micro Frontends promise to bring in a lot of benefits when it comes to structuring a complex application and simplifying its development, deployment and maintenance.
But there is a wonderful saying that goes “there is no one-size-fits-all approach that anyone can offer you. The same hot water that softens a carrot hardens an egg”. Micro Frontend is no silver bullet for your architectural problems and comes with its own share of downsides. With more repositories, more tools, more build/deploy pipelines, more servers, more domains to maintain, Micro Frontends can increase the complexity of an app. It may render cross-application communication difficult to establish. It can also lead to duplication of dependencies and an increase in application size.
Your decision to implement this architecture will depend on many factors like the size of your organization and the complexity of your application. Whether it is a new or legacy codebase, it is advisable to apply the technique gradually over time and review its efficacy over time.
GraphQL is becoming a popular way to use APIs in modern web and mobile apps.
However, learning new things is always time-consuming and without getting your hands dirty, it’s very difficult to understand the nuances of a new technology.
So, we have put together a powerful and concise tutorial that will guide you through setting up a GraphQL backend and integration into your React app in the shortest time possible. This tutorial is light on opinions, so that once you get a hang of the fundamentals, you can go on and tailor your workflow.
Key topics and takeaways:
Authentication
GraphQL API with AWS AppSync
Hosting
Working with multiple environments
Removing services
What will we be building?
We will build a basic real-time Restaurant CRUD app using authenticated GraphQL APIs. Click here to try the deployed version of the app to see what we’ll be building.
Will this tutorial teach React or GraphQL concepts as well?
No. The focus is to learn how to use AWS Amplify to build cloud-enabled, real-time web applications. If you are new to React or GraphQL, we recommend going through the official documentation and then coming back here.
If you have initialized the app with Typescript and see errors while using
aws-amplify-react, add aws-amplify-react.d.ts to src with:
declaremodule'aws-amplify-react';
Installing the AWS Amplify CLI and adding it to the project
To install the CLI:
npm install -g @aws-amplify/cli
Now we need to configure the CLI with our credentials:
amplify configure
If you’d like to see a video walkthrough of this process, click here
Here we’ll walk you through the amplify configure setup. After you sign in to the AWS console, follow these steps:
Specify the AWS region: ap-south-1 (Mumbai) <Select the region based on your location. Click here for reference>
Specify the username of the new IAM user: amplify-app <name of=”” your=”” app=””></name>
In the AWS Console, click Next: Permissions, Next: Tags, Next: Review, and Create User to create the new IAM user. Then, return to the command line and press Enter.
Enter the credentials of the newly created user: accessKeyId: <your_access_key_id> </your_access_key_id> secretAccessKey: <your_secret_access_key></your_secret_access_key>
Profile Name: default
To view the newly created IAM user, go to the dashboard. Also, make sure that your region matches your selection.
To add amplify to your project:
amplify init
Answer the following questions:
Enter a name for the project: amplify-app <name of=”” your=”” app=””></name>
Enter a name for the environment: dev <name of=”” your=”” environment=””></name>
Choose your default editor: Visual Studio Code <your default editor=””></your>
Choose the type of app that you’re building: javaScript
Please choose the profile you want to use: default
Now, the AWS Amplify CLI has initialized a new project and you will see a new folder: amplify. This folder has files that hold your project configuration.
Do you want to use default authentication and security configuration: Default configuration
How do you want users to be able to sign in when using your Cognito User Pool: Username
What attributes are required for signing up: Email
Now, let’s run the push command to create the cloud resources in our AWS account:
amplify push
To quickly check your newly created Cognito User Pool, you can run
amplify status
To access the AWS Cognito Console at any time, go to the dashboard. Also, ensure that your region is set correctly.
Now, our resources are created and we can start using them.
The first thing is to connect our React application to our new AWS Amplify project. To do this, reference the auto-generated aws-exports.js file that is now in our src folder.
To configure the app, open App.tsx and add the following code below the last import:
Now, we can start using our AWS services. To add the Authentication flow to the UI, export the app component by wrapping it with the authenticator HOC:
Now, let’s run the app to check if an Authentication flow has been added before our App component is rendered.
This flow gives users the ability to sign up and sign in. To view any users that were created, go back to the Cognito dashboard. Alternatively, you can also use:
amplify console auth
The withAuthenticator HOC is a really easy way to get up and running with authentication, but in a real-world application, we probably want more control over how our form looks and functions. We can use the aws-amplify/Auth class to do this. This class has more than 30 methods including signUp, signIn, confirmSignUp, confirmSignIn, and forgotPassword. These functions return a promise, so they need to be handled asynchronously.
Adding and Integrating the GraphQL API
To add GraphQL API, use the following command:
amplify add api
Answer the following questions:
Please select from one of the below mentioned services: GraphQL
Provide API name: RestaurantAPI
Choose an authorization type for the API: API key
Do you have an annotated GraphQL schema: No
Do you want a guided schema creation: Yes
What best describes your project: Single object with fields (e.g., “Todo” with ID, name, description)
Do you want to edit the schema now: Yes
When prompted, update the schema to the following:
Next, let’s run the push command to create the cloud resources in our AWS account:
amplify push
Are you sure you want to continue: Yes
Do you want to generate code for your newly created GraphQL API: Yes
Choose the code generation language target: typescript
Enter the file name pattern of graphql queries, mutations and subscriptions: src/graphql/**/*.ts
Do you want to generate/update all possible GraphQL operations – queries, mutations and subscriptions: Yes
Enter maximum statement depth [increase from default if your schema is deeply nested]: 2
Enter the file name for the generated code: src/API.ts
Notice your GraphQL endpoint and API KEY. This step has created a new AWS AppSync API and generated the GraphQL queries, mutations, and subscriptions on your local. To check, see src/graphql or visit the AppSync dashboard. Alternatively, you can use:
amplify console api
Please select from one of the below mentioned services: GraphQL
Now, in the AppSync console, on the left side click on Queries. Execute the following mutation to create a restaurant in the API:
mutation createRestaurant {createRestaurant(input: { name: "Nobu" description: "Great Sushi" city: "New York" }) { id name description city }}
Now, let’s query for the restaurant:
query listRestaurants { listRestaurants { items { id name description city } }}
We can even search / filter data when querying:
query searchRestaurants {listRestaurants(filter: { city: { contains: "New York" } }) { items { id name description city } }}
Now that the GraphQL API is created, we can begin interacting with it from our client application. Here is how we’ll add queries, mutations, and subscriptions:
Finally, we have our app ready. You can now sign-up,sign-in, add new restaurants, see real-time updates of newly added restaurants.
Hosting
The hosting category enables you to deploy and host your app on AWS.
amplify add hosting
Select the environment setup: DEV (S3 only with HTTP)
hosting bucket name: <YOUR_BUCKET_NAME>
index doc for the website: index.html
error doc for the website: index.html
Now, everything is set up & we can publish it:
amplify publish
Working with multiple environments
You can create multiple environments for your application to create & test out new features without affecting the main environment which you are working on.
When you use an existing environment to create a new environment, you get a copy of the entire backend application stack (CloudFormation) for the current environment. When you make changes in the new environment, you are then able to test these new changes in the new environment & merge only the changes that have been made since the new environment was created.
Let’s take a look at how to create a new environment. In this new environment, we’ll add another field for the restaurant owner to the GraphQL Schema.
First, we’ll initialize a new environment using amplify init:
amplify init
Do you want to use an existing environment: N
Enter a name for the environment: apiupdate
Do you want to use an AWS profile: Y
Once the new environment is initialized, we should be able to see some information about our environment setup by running:
amplify env list| Environments ||------------|| dev ||*apiupdate |
Now, add the owner field to the GraphQL Schema in
amplify/backend/api/RestaurantAPI/schema.graphql:
typeRestaurant @model { ... owner: String}
Run the push command to create a new stack:
amplify push.
After testing it out, it can be merged into our original dev environment:
Do you want to update code for your updated GraphQL API: Y
Do you want to generate GraphQL statements: Y
Removing Services
If at any time, you would like to delete a service from your project & your account, you can do this by running the amplify remove command:
amplify remove authamplify push
If you are unsure of what services you have enabled at any time, amplify status will give you the list of resources that are currently enabled in your app.
Sample code
The sample code for this blog post with an end to end working app is available here.
Summary
Once you’ve worked through all the sections above, your app should now have all the capabilities of a modern app, and building GraphQL + React apps should now be easier and faster with Amplify.
This blog aims at exploring the Rasa Stack to create a stateless chat-bot. We will look into how, the recently released Rasa Core, which provides machine learning based dialogue management, helps in maintaining the context of conversations using machine learning in an efficient way.
If you have developed chatbots, you would know how hopelessly bots fail in maintaining the context once complex use-cases need to be developed. There are some home-grown approaches that people currently use to build stateful bots. The most naive approach is to create the state machines where you create different states and based on some logic take actions. As the number of states increases, more levels of nested logic are required or there is a need to add an extra state to the state machine, with another set of rules for how to get in and out of that state. Both of these approaches lead to fragile code that is harder to maintain and update. Anyone who’s built and debugged a moderately complex bot knows this pain.
After building many chatbots, we have experienced that flowcharts are useful for doing the initial design of a bot and describing a few of the known conversation paths, but we shouldn’t hard-code a bunch of rules since this approach doesn’t scale beyond simple conversations.
Thanks to the Rasa guys who provided a way to go stateless where scaling is not at all a problem. Let’s build a bot using Rasa Core and learn more about this.
Rasa Core: Getting Rid of State Machines
The main idea behind Rasa Core is that thinking of conversations as a flowchart and implementing them as a state machine doesn’t scale. It’s very hard to reason about all possible conversations explicitly, but it’s very easy to tell, mid-conversation, if a response is right or wrong. For example, let’s consider a term insurance purchase bot, where you have defined different states to take different actions. Below diagram shows an example state machine:
Let’s consider a sample conversation where a user wants to compare two policies listed by policy_search state.
In above conversation, it can be compared very easily by adding some logic around the intent campare_policies. But real life is not so easy, as a majority of conversations are edge cases. We need to add rules manually to handle such cases, and after testing we realize that these clash with other rules we wrote earlier.
Rasa guys figured out how machine learning can be used to solve this problem. They have released Rasa Core where the logic of the bot is based on a probabilistic model trained on real conversations.
Structure of a Rasa Core App
Let’s understand few terminologies we need to know to build a Rasa Core app:
1. Interpreter: An interpreter is responsible for parsing messages. It performs the Natural Language Understanding and transforms the message into structured output i.e. intent and entities. In this blog, we are using Rasa NLU model as an interpreter. Rasa NLU comes under the Rasa Stack. In Training section, it is shown in detail how to prepare the training data and create a model.
2. Domain: To define a domain we create a domain.yml file, which defines the universe of your bot. Following things need to be defined in a domain file:
Intents: Things we expect the user to say. It is more related to Rasa NLU.
Entities: These represent pieces of information extracted what user said. It is also related to Rasa NLU.
Templates: We define some template strings which our bot can say. The format for defining a template string is utter_<intent>. These are considered as actions which bot can take.
Actions: List of things bot can do and say. There are two types of actions we define one those which will only utter message (Templates) and others some customised actions where some required logic is defined. Customised actions are defined as Python classes and are referenced in domain file.
Slots: These are user-defined variables which need to be tracked in a conversation. For e.g to buy a term insurance we need to keep track of what policy user selects and details of the user, so all these details will come under slots.
3. Stories: In stories, we define what bot needs to do at what point in time. Based on these stories, a probabilistic model is generated which is used to decide which action to be taken next. There are two ways in which stories can be created which are explained in next section.
Let’s combine all these pieces together. When a message arrives in a Rasa Core app initially, interpreter transforms the message into structured output i.e. intents and entities. The Tracker is the object which keeps track of conversation state. It receives the info that a new message has come in. Then based on dialog model we generate using domain and stories policy chooses which action to take next. The chosen action is logged by the tracker and response is sent back to the user.
Training and Running A Sample Bot
We will create a simple Facebook chat-bot named Secure Life which assists you in buying term life insurance. To keep the example simple, we have restricted options such as age-group, term insurance amount, etc.
There are two models we need to train in the Rasa Core app:
Rasa NLU model based on which messages will be processed and converted to a structured form of intent and entities. Create following two files to generate the model:
data.json: Create this training file using the rasa-nlu trainer. Click here to know more about the rasa-nlu trainer.
$ python -m rasa_nlu.train -c nlu_model_config.json --fixed_model_name current
Dialogue Model: This model is trained on stories we define, based on which the policy will take the action. There are two ways in which stories can be generated:
Supervised Learning: In this type of learning we will create the stories by hand, writing them directly in a file. It is easy to write but in case of complex use-cases it is difficult to cover all scenarios.
Reinforcement Learning: The user provides feedback on every decision taken by the policy. This is also known as interactive learning. This helps in including edge cases which are difficult to create by hand. You must be thinking how it works? Every time when a policy chooses an action to take, it is asked from the user whether the chosen action is correct or not. If the action taken is wrong, you can correct the action on the fly and store the stories to train the model again.
Since the example is simple, we have used supervised learning method, to generate the dialogue model. Below is the stories.md file.
## All yes* greet- utter_greet* affirm- utter_very_much_so* affirm- utter_gender* gender- utter_coverage_duration- action_gender* affirm- utter_nicotine* affirm- action_nicotine* age- action_thanks## User not interested* greet- utter_greet* deny- utter_decline## Coverage duration is not sufficient* greet- utter_greet* affirm- utter_very_much_so* affirm- utter_gender* gender- utter_coverage_duration- action_gender* deny- utter_decline
Run below command to train dialogue model :
$ python -m rasa_core.train -s <path to stories.md file>-d <path to domain.yml>-o models/dialogue --epochs 300
Define a Domain: Create domain.yml file containing all the required information. Among the intents and entities write all those strings which bot is supposed to see when user say something i.e. intents and entities you defined in rasa NLU training file.
intents:- greet- goodbye- affirm- deny- age- genderslots:gender:type: textnicotine:type: textagegroup:type: texttemplates:utter_greet:-"hey there! welcome to Secure-Life!\nI can help you quickly estimate your rate of coverage.\nWould you like to do that ?"utter_very_much_so:-"Great! Let's get started.\nWe currently offer term plans of Rs. 1Cr. Does that suit your need?"utter_gender:-"What gender do you go by ?"utter_coverage_duration:-"We offer this term plan for a duration of 30Y. Do you think that's enough to cover entire timeframe of your financial obligations ?"utter_nicotine:-"Do you consume nicotine-containing products?"utter_age:-"And lastly, how old are you ?"utter_thanks:-"Thank you for providing all the info. Let me calculate the insurance premium based on your inputs."utter_decline:-"Sad to see you go. In case you change your plans, you know where to find me :-)"utter_goodbye:-"goodbye :("actions:- utter_greet- utter_goodbye- utter_very_much_so- utter_coverage_duration- utter_age- utter_nicotine- utter_gender- utter_decline- utter_thanks- actions.ActionGender- actions.ActionNicotine- actions.ActionThanks
Define Actions: Templates defined in domain.yml also considered as actions. A sample customized action is shown below where we are setting a slot named gender with values according to the option selected by the user.
from rasa_core.actions.action import Actionfrom rasa_core.events import SlotSetclass ActionGender(Action):def name(self):return 'action_gender'def run(self, dispatcher, tracker, domain):messageObtained = tracker.latest_message.text.lower()if ("male" in messageObtained):return [SlotSet("gender", "male")]elif ("female" in messageObtained):return [SlotSet("gender", "female")]else:return [SlotSet("gender", "others")]
Running the Bot
Create a Facebook app and get the app credentials. Create a bot.py file as shown below:
from rasa_core import utilsfrom rasa_core.agent import Agentfrom rasa_core.interpreter import RasaNLUInterpreterfrom rasa_core.channels import HttpInputChannelfrom rasa_core.channels.facebook import FacebookInputlogger = logging.getLogger(__name__)def run(serve_forever=True):# create rasa NLU interpreterinterpreter = RasaNLUInterpreter("models/nlu/current")agent = Agent.load("models/dialogue", interpreter=interpreter)input_channel = FacebookInput(fb_verify="your_fb_verify_token", # you need tell facebook this token, to confirm your URLfb_secret="your_app_secret", # your app secretfb_tokens={"your_page_id": "your_page_token"}, # page ids + tokens you subscribed todebug_mode=True # enable debug mode for underlying fb library)if serve_forever:agent.handle_channel(HttpInputChannel(5004, "/app", input_channel))return agentif __name__ == '__main__':utils.configure_colored_logging(loglevel="DEBUG")run()
Run the file and your bot is ready to test. Sample conversations are provided below:
Summary
You have seen how Rasa Core has made it easier to build bots. Just create few files and boom! Your bot is ready! Isn’t it exciting? I hope this blog provided you some insights on how Rasa Core works. Start exploring and let us know if you need any help in building chatbots using Rasa Core.
Building a React application is not only about creating a user interface. It also has tricky parts like data fetching, re-render performance, and scalability. Many libraries and frameworks try to solve these problems, like Redux, Sagas, etc. But these tools come with their own set of difficulties.
Redux gives you a single data source, but all the data fetching and rendering logic is handled by developers. Immer gives you immutable data structures, but one needs to handle the re-render performance of applications.
GraphQL helps developers design and expose APIs on the backend, but no tool on the client side could utilize the full advantage of the single endpoint and data schema provided by GraphQL.
In this article, we will learn about Relay as a GraphQL client. What are the advantages of using Relay in your application, and what conventions are required to integrate it? We’ll also cover how following those conventions will give you a better developer experience and a performant app. We will also see how applications built with Relay are modular, scalable, efficient, and, by default, resilient to change.
About Relay
Relay is a JavaScript framework to declaratively fetch and manage your GraphQL data inside a React application. Relay uses static queries and ahead-of-time compilation to help you build a high-performance app.
But as the great saying goes, “With great power comes great responsibilities.” Relay comes with a set of costs (conventions), which—when compared with the benefits you get—is well worth it. We will explore the trade-offs in this article.
The Relay framework is built of multiple modules:
1. The compiler: This is a set of modules designed to extract GraphQL code from across the codebase and do validations and optimizations during build time.
2. Relay runtime: A high-performance GraphQL runtime that features a normalized cache for objects and highly optimized read/write operations, simplified abstractions over fetching data fields, garbage collection, subscriptions, and more.
3. React-relay: This provides the high-level APIs to integrate React with the Relay runtime.
The Relay compiler runs as a separate process, like how webpack works for React. It keeps watching and compiling the GraphQL code, and in case of errors, it simply does not build your code, which prevents bugs from going into higher environments.
Fragments
Fragments are at the heart of how Relay blends with GraphQL. A fragment is a selection of fields on a GraphQL type.
fragment Avatar_user on User { avatarImgUrl firstName lastName userName}
If we look at the sample fragment definition above, the fragment name, Avatar_user, is not just a random name. One of the Relay framework’s important conventions is that fragments have globally unique fragment names and follow a structure of <modulename>_<propertyname>. The example above is a fragment definition for Avatar_user.</propertyname></modulename>
This fragment can then be reused throughout the queries instead of selecting the fields manually to render the avatar in each view.
In the below query, we see the author type, and the first two who liked the blog post can use the fragment definition of Avatar_user
Fragments not only allow us to reuse the definitions but more essentially, they let us add or remove fields needed to render our avatar as we evolve our application.
Another highly important client-side convention is colocation. This means the data required for a component lives inside the component. This makes maintenance and extending much easier. Just like how React allows us to break our UI elements into components and group/compose different views, fragments in Relay allow us to split the data definitions and colocate the data and the view definitions.
So, a good practice is to define single or multiple fragments that contain the data component to be rendered. This means that a component depends on some fields from the user type, irrespective of the parent component. In the example above, the <avatar> component will render an avatar using the fields specified in the Avatar_user fragment named.</avatar>
How Relay leverages the GraphQL Fragment
Relay wants all components to enlist all the data it needs to render, along with the component itself. Relay uses data and fragments to integrate the component and its data requirement. This convention mandates that every component lists the fields it needs access to.
Other advantages of the above are:
Components are not dependent on data they don’t explicitly request.
Components are modular and self-contained.
Reusing and refactoring the components becomes easier.
Performance
In Relay, the component re-renders only when its exact fields change, and this feature available is out of the box. The fragment subscribes to updates specifically for data the component selects. This lets Relay enhance how the view is updated, and performance is not affected as codebase scales.
Now, let’s look at an example of components in a single post of a blog application. Here is a wireframe of a sample post to give an idea of the data and view required.
Now, let’s write a plain query without Relay, which will fetch all the data in a single query. It will look like this for the above wireframe:
In the implementation above, we have a single query that will be managed by the top-level component. It will be the top-level component’s responsibility to fetch the data and pass it down as props. Now, we will look at how we would build this in Relay:
The useLazyLoadQuery React hook from Relay will start fetching the GetBlogPost query just as the component renders.
NOTE: The useLazyLoadQuery is used here as it follows a common mental model of fetching data after the page is loaded. However, Relay encourages data to be fetched as early as possible using the usePreladedQuery hook.
For type safety, we are annotating the useLazyLoadQuery with the type GetBlogPost, which is imported from ./__generated__/GetBlogPost.graphql. This file is auto-generated and synced by the Relay compiler. It contains all the information about the types needed to be queried, along with the return type of data and the input variables for the query.
The Relay compiler takes all the declared fragments in the codebase and generates the type files, which can then be used to annotate a particular component.
The GetBlogPost query is defined by composing multiple fragments. Another great aspect of Relay is that there is no need to import the fragments manually. They are automatically included by the Relay compiler. Building the query by composing fragments, just like how we compose our component, is the key here.
Another approach can be to define queries per component, which takes full responsibility for its data requirements. But this approach has two problems:
1. Multiple queries are sent to the server instead of one.
2. The loading will be slower as components would have to wait till they render to start fetching the data.
In the above example, the GetBlogPost only deals with including the fragments for its child components, BlogPostHead and BlogPostBody. It is kept hidden from the actual data fields of the children component.
When using Relay, components define their data requirement by themselves. These components can then be composed along with other components that have their own separate data.
At the same time, no component knows what data the other component needs except from the GraphQL type that has the required component data. Relay makes sure the right data is passed to the respective component, and all input for a query is sent to the server.
This allows developers to think only about the component and fragments as one while Relay does all the heavy lifting in the background. Relay minimizes the round-trips to the server by placing the fragments from multiple components into optimized and efficient batches.
As we said earlier, the two fragments, BlogPostHead_blogPost and BlogPostBody_blogPost, which we referenced in the query, are not imported manually. This is because Relay imposes unique fragment names globally so that the compiler can include the definitions in queries sent to the server. This eliminates the chances of errors and takes away the laborious task of referencing the fragments by hand.
Now, in the rendering logic above, we render the <BlogPostHead/> and <BlogPostBody/> and pass the blogPostById object as prop. It’s passed because it is the object inside the query that spreads the fragment needed by the two components. This is how Relay transfers fragment data. Because we spread both fragments on this object, it is guaranteed to satisfy both components.
To put it into simpler terms, we say that to pass the fragment data, we pass the object where the fragment is spread, and the component then uses this object to get the real fragment data. Relay, through its robust type systems, makes sure that the right object is passed with required fragment spread on it.
The previous component, the BlogPost, was the Parent component, i.e., the component with the root query object. The root query is necessary because it cannot fetch a fragment in isolation. Fragments must be included in the root query in a parent component. The parent can, in turn, be a fragment as long the root query exists in the hierarchy. Now, we will build the BlogPostHead component using fragments:
NOTE: In our example, the BlogPostHead and BlogPostBody define only one fragment, but in general, a component can have any number of fragments or GraphQL types and even more than one fragments on the same type.
In the component above, two type definitions, namely BlogPostHead_blogPost$key and BlogPostHead_blogPost, are imported from the file BlogPostHead_blogPost.graphql, generated by the Relay compiler. The compiler extracts the fragment code from this file and generates the types. This process is followed for all the GraphQL code—queries, mutations, fragments, and subscriptions.
The blogPostHead_blogPost has the fragment type definitions, which is then passed to the useFragment hook to ensure type safety when using the data from the fragment. The other import, blogPostHead_blogPost$key, is used in the interface Props { … }, and this type definition makes sure that we pass the right object to useFragment. Otherwise, the type system will throw errors during build time. In the above child component, the blogPost object is received as a prop and is passed to useFragment as a second parameter. If the blogPost object did not have the correct fragment, i.e., BlogPostHead_blogPost, spread on it, we would have received a type error. Even if there were another fragment with exact same data selection spread on it, Relay makes sure it’s the right fragment that we use with the useFragement. This allows you to change the update fragment definitions without affecting other components.
Data masking
In our example, the fragment BlogPostHead_blogPost explicitly selects two fields for the component:
title
coverImgUrl
This is because we use/access only these two fields in the view for the <blogposthead></blogposthead> component. So, even if we define another fragment, BlogPostAuthor_blogPost, which selects the title and coverImgUrl, we don’t receive access to them unless we ask for them in the same fragment. This is enforced by Relay’s type system both at compile time and at runtime. This safety feature of Relay makes it impossible for components to depend on data they do not explicitly select. So, developers can refactor the components without risking other components. To reiterate, all components and their data dependencies are self-contained.
The data for this component, i.e., title and coverImgUrl, will not be accessible on the parent component, BlogPost, even though the props object is sent by the parent. The data becomes available only through the useFragment React hook. This hook can consume the fragment definition. The useFragment takes in the fragment definition and the object where the fragment is spread to get the data listed for the particular fragment.
Just like how we spread the fragment for the BlogPostHead component in the BlogPost root query, we an also extend this to the child components of BlogPostHead. We spread the fragments, i.e., BlogPostAuthor_blogPost, BlogPostLikeControls_blogPost, since we are rendering <BlogPostAuthor /> and <BlogPostLikeControls />.
NOTE: The useFragment hook does not fetch the data. It can be thought of as a selector that grabs only what is needed from the data definitions.
Performance
When using a fragment for a component, the component subscribes only to the data it depends on. In our example, the component BlogPostHead will only automatically re-render when the fields “coverImgUrl” or “title” change for a specific blog post the component renders. Since the BlogPostAuthor_blogPost fragment does not select those fields, it will not re-render. Subscription to any updates is made on fragment level. This is an essential feature that works out of the box with Relay for performance.
Let us now see how general data and components are updated in a different GraphQL framework than Relay. The data that gets rendered on view actually comes from an operation that requests data from the server, i.e., a query or mutation. We write the query that fetches data from the server, and that data is passed down to different components as per their needs as props. The data flows from the root component, i.e., the component with the query, down to the components.
Let’s look at a graphical representation of the data flow in other GraphQL frameworks:
For the initial part, we see nothing changes. We still have a query that is sent to the GraphQL server and the data is fetched and stored in the Relay data store.
What Relay does after this is different. The components get the data directly from the cache-store(data store). This is because the fragments help Relay integrate deeply with the component data requirements.The component fragments get the data straight from the framework data store and do not rely on data to be passed down as props. Although some information is passed from the query to the fragments used to look up the particular data needed from the data store, the data is fetched by the fragment itself.
To conclude the above comparison, in other frameworks (like Apollo), the component uses the query as the data source. The implementation details of how the root component executing the query sends data to its descendants is left to us. But Relay takes a different approach of letting the component take care of the data in needs from the data store.
In an approach used by other GraphQL frameworks, the query is the data source, and updates in the data store forces the component holding the query to re-render. This re-render cascades down to any number of components even if those components do not have to do anything with the updated data other than acting as a layer to pass data from parent to child. In the Relay approach, the components directly subscribe to the updates for the data used. This ensures the best performance as our app scales in size and complexity.
Developer Experience
Relay removes the responsibility of developers to route the data down from query to the components that need it. This eliminates the changes of developer error. There is no way for a component to accidentally or deliberately depend on data that it should be just passing down in the component tree if it cannot access it. All the hard work is taken care of by the Relay framework if we follow the conventions discussed.
Conclusion
To summarize, we detailed all the work Relay does for us and the effects:
The type system of the Relay framework makes sure the right components get the right data they need. Everything in Relay revolves around fragments.
In Relay, fragments are coupled and colocated with components, which allows it to mask the data requirements from the outside world. This increases the readability and modularity.
By default, Relay takes care of performance as components only re-render when the exact data they use change in the data store.
Type generation is a main feature of Relay compiler. Through type generation, interactions with the fragment’s data is typesafe.
Conventions enforced by Relay’s philosophy and architecture allows it to take advantage of the information available about your component. It knows the exact data dependencies and types. It uses all this information to do a lot of work that developers are required to deal with.
Asynchronous programming is a characteristic of modern programming languages that allows an application to perform various operations without waiting for any of them. Asynchronicity is one of the big reasons for the popularity of Node.js.
We have discussed Python’s asynchronous features as part of our previous post: an introduction to asynchronous programming in Python. This blog is a natural progression on the same topic. We are going to discuss async features in Python in detail and look at some hands-on examples.
Consider a traditional web scraping application that needs to open thousands of network connections. We could open one network connection, fetch the result, and then move to the next ones iteratively. This approach increases the latency of the program. It spends a lot of time opening a connection and waiting for others to finish their bit of work.
On the other hand, async provides you a method of opening thousands of connections at once and swapping among each connection as they finish and return their results. Basically, it sends the request to a connection and moves to the next one instead of waiting for the previous one’s response. It continues like this until all the connections have returned the outputs.
From the above chart, we can see that using synchronous programming on four tasks took 45 seconds to complete, while in asynchronous programming, those four tasks took only 20 seconds.
Where Does Asynchronous Programming Fit in the Real-world?
Asynchronous programming is best suited for popular scenarios such as:
1. The program takes too much time to execute.
2. The reason for the delay is waiting for input or output operations, not computation.
3. For the tasks that have multiple input or output operations to be executed at once.
And application-wise, these are the example use cases:
Web Scraping
Network Services
Difference Between Parallelism, Concurrency, Threading, and Async IO
Because we discussed this comparison in detail in our previous post, we will just quickly go through the concept as it will help us with our hands-on example later.
Parallelism involves performing multiple operations at a time. Multiprocessing is an example of it. It is well suited for CPU bound tasks.
Concurrency is slightly broader than Parallelism. It involves multiple tasks running in an overlapping manner.
Threading – a thread is a separate flow of execution. One process can contain multiple threads and each thread runs independently. It is ideal for IO bound tasks.
Async IO is a single-threaded, single-process design that uses cooperative multitasking. In simple words, async IO gives a feeling of concurrency despite using a single thread in a single process.
Fig:- A comparison in concurrency and parallelism
Components of Async IO Programming
Let’s explore the various components of Async IO in depth. We will also look at an example code to help us understand the implementation.
1. Coroutines
Coroutines are mainly generalization forms of subroutines. They are generally used for cooperative tasks and behave like Python generators.
An async function uses the await keyword to denote a coroutine. When using the await keyword, coroutines release the flow of control back to the event loop.
To run a coroutine, we need to schedule it on the event loop. After scheduling, coroutines are wrapped in Tasks as a Future object.
Example:
In the below snippet, we called async_func from the main function. We have to add the await keyword while calling the sync function. As you can see, async_func will do nothing unless the await keyword implementation accompanies it.
import asyncioasync def async_func(): print('Velotio ...') await asyncio.sleep(1) print('... Technologies!')async def main(): async_func()#this will do nothing because coroutine object is created but not awaited await async_func()asyncio.run(main())
Output
RuntimeWarning: coroutine 'async_func' was never awaitedasync_func()#this will do nothing because coroutine object is created but not awaitedRuntimeWarning: Enable tracemalloc to get the object allocation tracebackVelotio ...... Blog!
2. Tasks
Tasks are used to schedule coroutines concurrently.
When submitting a coroutine to an event loop for processing, you can get a Task object, which provides a way to control the coroutine’s behavior from outside the event loop.
Example:
In the snippet below, we are creating a task using create_task (an inbuilt function of asyncio library), and then we are running it.
This mechanism runs coroutines until they complete. You can imagine it as while(True) loop that monitors coroutine, taking feedback on what’s idle, and looking around for things that can be executed in the meantime.
It can wake up an idle coroutine when whatever that coroutine is waiting on becomes available.
Only one event loop can run at a time in Python.
Example:
In the snippet below, we are creating three tasks and then appending them in a list and executing all tasks asynchronously using get_event_loop, create_task and the await function of the asyncio library.
A future is a special, low-level available object that represents an eventual result of an asynchronous operation.
When a Future object is awaited, the co-routine will wait until the Future is resolved in some other place.
We will look into the sample code for Future objects in the next section.
A Comparison Between Multithreading and Async IO
Before we get to Async IO, let’s use multithreading as a benchmark and then compare them to see which is more efficient.
For this benchmark, we will be fetching data from a sample URL (the Velotio Career webpage) with different frequencies, like once, ten times, 50 times, 100 times, 500 times, respectively.
We will then compare the time taken by both of these approaches to fetch the required data.
Implementation
Code of Multithreading:
import requestsimport timefrom concurrent.futures import ProcessPoolExecutordef fetch_url_data(pg_url): try: resp = requests.get(pg_url) except Exception as e: print(f"Error occured during fetch data from url{pg_url}") else: return resp.contentdef get_all_url_data(url_list): with ProcessPoolExecutor() as executor: resp = executor.map(fetch_url_data, url_list) return respif __name__=='__main__': url = "https://www.velotio.com/careers" for ntimes in [1,10,50,100,500]: start_time = time.time() responses = get_all_url_data([url] * ntimes) print(f'Fetch total {ntimes} urls and process takes {time.time() - start_time} seconds')
Output
Fetch total 1 urls and process takes 1.8822264671325684 secondsFetch total 10 urls and process takes 2.3358211517333984 secondsFetch total 50 urls and process takes 8.05638575553894 secondsFetch total 100 urls and process takes 14.43302869796753 secondsFetch total 500 urls and process takes 65.25404500961304 seconds
ProcessPoolExecutor is a Python package that implements the Executor interface. The fetch_url_data is a function to fetch the data from the given URL using the requests python package, and the get_all_url_data function is used to map the fetch_url_data function to the lists of URLs.
Async IO Programming Example:
import asyncioimport timefrom aiohttp import ClientSession, ClientResponseErrorasync def fetch_url_data(session, url): try: async with session.get(url, timeout=60) as response: resp = await response.read() except Exception as e: print(e) else: return resp returnasync def fetch_async(loop, r): url = "https://www.velotio.com/careers" tasks = [] async with ClientSession() as session: for i in range(r): task = asyncio.ensure_future(fetch_url_data(session, url)) tasks.append(task) responses = await asyncio.gather(*tasks) return responsesif __name__ == '__main__': for ntimes in [1, 10, 50, 100, 500]: start_time = time.time() loop = asyncio.get_event_loop() future = asyncio.ensure_future(fetch_async(loop, ntimes)) loop.run_until_complete(future) #will run until it finish or get any error responses = future.result() print(f'Fetch total {ntimes} urls and process takes {time.time() - start_time} seconds')
Output
Fetch total 1 urls and process takes 1.3974951362609863 secondsFetch total 10 urls and process takes 1.4191942596435547 secondsFetch total 50 urls and process takes 2.6497368812561035 secondsFetch total 100 urls and process takes 4.391665458679199 secondsFetch total 500 urls and process takes 4.960426330566406 seconds
We need to use the get_event_loop function to create and add the tasks. For running more than one URL, we have to use ensure_future and gather function.
The fetch_async function is used to add the task in the event_loop object and the fetch_url_data function is used to read the data from the URL using the session package. The future_result method returns the response of all the tasks.
Results:
As you can see from the plot, async programming is much more efficient than multi-threading for the program above.
The graph of the multithreading program looks linear, while the asyncio program graph is similar to logarithmic.
Conclusion
As we saw in our experiment above, Async IO showed better performance with the efficient use of concurrency than multi-threading.
Async IO can be beneficial in applications that can exploit concurrency. Though, based on what kind of applications we are dealing with, it is very pragmatic to choose Async IO over other implementations.
We hope this article helped further your understanding of the async feature in Python and gave you some quick hands-on experience using the code examples shared above.
In the last few years, we saw a great shift in technology, where projects are moving towards “microservice architecture” vs the old “monolithic architecture”. This approach has done wonders for us.
As we say, “smaller things are much easier to handle”, so here we have microservices that can be handled conveniently. We need to interact among different microservices. I handled it using the HTTP API call, which seems great and it worked for me.
But is this the perfect way to do things?
The answer is a resounding, “no,” because we compromised both speed and efficiency here.
Then came in the picture, the gRPC framework, that has been a game-changer.
“gRPC or Google Remote Procedure Call is a modern open-source high-performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication.”
RPC or remote procedure calls are the messages that the server sends to the remote system to get the task(or subroutines) done.
Google’s RPC is designed to facilitate smooth and efficient communication between the services. It can be utilized in different ways, such as:
Efficiently connecting polyglot services in microservices style architecture
Connecting mobile devices, browser clients to backend services
Generating efficient client libraries
Why gRPC?
– HTTP/2 based transport – It uses HTTP/2 protocol instead of HTTP 1.1. HTTP/2 protocol provides multiple benefits over the latter. One major benefit is multiple bidirectional streams that can be created and sent over TCP connections parallelly, making it swift.
– Auth, tracing, load balancing and health checking – gRPC provides all these features, making it a secure and reliable option to choose.
– Language independent communication– Two services may be written in different languages, say Python and Golang. gRPC ensures smooth communication between them.
– Use of Protocol Buffers – gRPC uses protocol buffers for defining the type of data (also called Interface Definition Language (IDL)) to be sent between the gRPC client and the gRPC server. It also uses it as the message interchange format.
Let’s dig a little more into what are Protocol Buffers.
Protocol Buffers
Protocol Buffers like XML, are an efficient and automated mechanism for serializing structured data. They provide a way to define the structure of data to be transmitted. Google says that protocol buffers are better than XML, as they are:
simpler
three to ten times smaller
20 to 100 times faster
less ambiguous
generates data access classes that make it easier to use them programmatically
Protobuf are defined in .proto files. It is easy to define them.
Types of gRPC implementation
1. Unary RPCs:- This is a simple gRPC which works like a normal function call. It sends a single request declared in the .proto file to the server and gets back a single response from the server.
2. Server streaming RPCs:- The client sends a message declared in the .proto file to the server and gets back a stream of message sequence to read. The client reads from that stream of messages until there are no messages.
3. Client streaming RPCs:- The client writes a message sequence using a write stream and sends the same to the server. After all the messages are sent to the server, the client waits for the server to read all the messages and return a response.
4. Bidirectional streaming RPCs:- Both gRPC client and the gRPC server use a read-write stream to send a message sequence. Both operate independently, so gRPC clients and gRPC servers can write and read in any order they like, i.e. the server can read a message then write a message alternatively, wait to receive all messages then write its responses, or perform reads and writes in any other combination.
**gRPC guarantees the ordering of messages within an individual RPC call. In the case of Bidirectional streaming, the order of messages is preserved in each stream.
Implementing gRPC in Python
Currently, gRPC provides support for many languages like Golang, C++, Java, etc. I will be focussing on its implementation using Python.
This will install all the required dependencies to implement gRPC.
Unary gRPC
For implementing gRPC services, we need to define three files:-
Proto file – Proto file comprises the declaration of the service that is used to generate stubs (<package_name>_pb2.py and <package_name>_pb2_grpc.py). These are used by the gRPC client and the gRPC server.</package_name></package_name>
gRPC client – The client makes a gRPC call to the server to get the response as per the proto file.
gRPC Server – The server is responsible for serving requests to the client.
syntax ="proto3";package unary;service Unary{// A simple RPC.//// Obtains the MessageResponse at a given position. rpc GetServerResponse(Message) returns (MessageResponse) {}}message Message{ string message =1;}message MessageResponse{ string message =1; bool received =2;}
In the above code, we have declared a service named Unary. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a MessageResponse. Below the service declaration, I have declared Message and Message Response.
Once we are done with the creation of the .proto file, we need to generate the stubs. For that, we will execute the below command:-
Two files are generated named unary_pb2.py and unary_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and the client.
Implementing the Server
import grpcfrom concurrent import futuresimport timeimport unary.unary_pb2_grpc as pb2_grpcimport unary.unary_pb2 as pb2class UnaryService(pb2_grpc.UnaryServicer): def __init__(self, *args, **kwargs): pass def GetServerResponse(self, request, context): # get the string from the incoming request message = request.message result = f'Hello I am up and running received "{message}" message from you' result = {'message': result, 'received': True}return pb2.MessageResponse(**result)def serve(): server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) pb2_grpc.add_UnaryServicer_to_server(UnaryService(), server) server.add_insecure_port('[::]:50051') server.start() server.wait_for_termination()if __name__ =='__main__':serve()
In the gRPC server file, there is a GetServerResponse() method which takes `Message` from the client and returns a `MessageResponse` as defined in the proto file.
server() function is called from the main function, and makes sure that the server is listening to all the time. We will run the unary_server to start the server
python3 unary_server.py
Implementing the Client
import grpcimport unary.unary_pb2_grpc as pb2_grpcimport unary.unary_pb2 as pb2class UnaryClient(object):""" Client for gRPC functionality""" def __init__(self): self.host = 'localhost' self.server_port = 50051 # instantiate a channel self.channel = grpc.insecure_channel('{}:{}'.format(self.host, self.server_port)) # bind the client and the server self.stub = pb2_grpc.UnaryStub(self.channel) def get_url(self, message):""" Client function to call the rpc for GetServerResponse""" message = pb2.Message(message=message) print(f'{message}') return self.stub.GetServerResponse(message)if __name__ == '__main__': client = UnaryClient() result = client.get_url(message="Hello Server you there?") print(f'{result}')
In the __init__func. we have initialized the stub using ` self.stub = pb2_grpc.UnaryStub(self.channel)’ And we have a get_url function which calls to server using the above-initialized stub
This completes the implementation of Unary gRPC service.
Let’s check the output:-
Run -> python3 unary_client.py
Output:-
message: “Hello Server you there?”
message: “Hello I am up and running. Received ‘Hello Server you there?’ message from you”
received: true
Bidirectional Implementation
syntax ="proto3";package bidirectional;service Bidirectional {// A Bidirectional streaming RPC.//// Accepts a stream of Message sent while a route is being traversed, rpc GetServerResponse(stream Message) returns (stream Message) {}}message Message { string message =1;}
In the above code, we have declared a service named Bidirectional. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a Message. Below the service declaration, I have declared Message.
Once we are done with the creation of the .proto file, we need to generate the stubs. To generate the stub, we need the execute the below command:-
Two files are generated named bidirectional_pb2.py and bidirectional_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and client.
Implementing the Server
from concurrent import futuresimport grpcimport bidirectional.bidirectional_pb2_grpc as bidirectional_pb2_grpcclass BidirectionalService(bidirectional_pb2_grpc.BidirectionalServicer): def GetServerResponse(self, request_iterator, context): for message in request_iterator: yield messagedef serve(): server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) bidirectional_pb2_grpc.add_BidirectionalServicer_to_server(BidirectionalService(), server) server.add_insecure_port('[::]:50051') server.start() server.wait_for_termination()if __name__ == '__main__': serve()
In the gRPC server file, there is a GetServerResponse() method which takes a stream of `Message` from the client and returns a stream of `Message` independent of each other. server() function is called from the main function and makes sure that the server is listening to all the time.
We will run the bidirectional_server to start the server:
python3 bidirectional_server.py
Implementing the Client
from __future__ import print_functionimport grpcimport bidirectional.bidirectional_pb2_grpc as bidirectional_pb2_grpcimport bidirectional.bidirectional_pb2 as bidirectional_pb2def make_message(message): return bidirectional_pb2.Message( message=message )def generate_messages(): messages = [ make_message("First message"), make_message("Second message"), make_message("Third message"), make_message("Fourth message"), make_message("Fifth message"), ] for msg in messages: print("Hello Server Sending you the %s" % msg.message) yield msgdef send_message(stub): responses = stub.GetServerResponse(generate_messages()) for response in responses: print("Hello from the server received your %s" % response.message)def run(): with grpc.insecure_channel('localhost:50051') as channel: stub = bidirectional_pb2_grpc.BidirectionalStub(channel) send_message(stub)if __name__ == '__main__': run()
In the run() function. we have initialised the stub using ` stub = bidirectional_pb2_grpc.BidirectionalStub(channel)’
And we have a send_message function to which the stub is passed and it makes multiple calls to the server and receives the results from the server simultaneously.
This completes the implementation of Bidirectional gRPC service.
Let’s check the output:-
Run -> python3 bidirectional_client.py
Output:-
Hello Server Sending you the First message
Hello Server Sending you the Second message
Hello Server Sending you the Third message
Hello Server Sending you the Fourth message
Hello Server Sending you the Fifth message
Hello from the server received your First message
Hello from the server received your Second message
Hello from the server received your Third message
Hello from the server received your Fourth message
Hello from the server received your Fifth message
gRPC is an emerging RPC framework that makes communication between microservices smooth and efficient. I believe gRPC is currently confined to inter microservice but has many other utilities that we will see in the coming years. To know more about modern data communication solutions, check out this blog.
In this post, we will talk about building a collaborative recommendation system. For this, we will utilize patient ratings with a drug and medical condition dataset to generate treatment suggestions.
Let’s take a practical scenario where multiple medical practitioners have treated patients with different medical conditions with the most suitable drugs available. For every prescribed drug, the patients are diagnosed and then suggested a treatment plan, which is our experiences.
The purpose of the recommendation system is to understand and find patterns with the information provided by patients during the diagnosis, and then suggest a treatment plan, which most closely matches the pattern identified by the recommendation system.
At the end of this article, we are going deeper into how these recommendations work and how we can find one preferred suggestion and the next five closest suggestions for any treatment.
Definitions
A recommendation system suggests or predicts a user’s behaviour by observing patterns of their past behaviour compared to others.
In simple terms, it is a filtering engine that picks more relevant information for specific users by using all the available information. It is often used in ecommerce like Amazon, Flipkart, Youtube, and Netflix and personalized user products like Alexa and Google Home Mini.
For the medical industry, where suggestions must be most accurate, a recommendation system will also take experiences into account. So, we must use all our experiences, and such applications will use every piece of information for any treatment.
Recommendation systems use information like various medical conditions and their effect on each patient. They compare these patterns to every new treatment to find the closest similarity.
Concepts and Technology
To design the recommendation system, we need a few concepts, which are listed below.
As far as the prototype development is concerned, we have support of a library (Scipy & Sklearn) that executes all the algorithms for us. All we need is a little Python and to use library functions.
Different Approaches for Recommendation Systems
Below I have listed a few filtering approaches and examples:
Collaborative filtering: It is based on review or response of users for any entity. Here, the suggestion is based on the highest rated item by most of the users. E.g., movie or mobile suggestions.
Content-based filtering: It is based on the pattern of each user’s past activity. Here, the suggestion is based on the most preferred by similar users. E.g., food suggestions.
Popularity-based filtering: It is based on a pattern of popularity among all users. E.g., YouTube video suggestions
Based on these filtering approaches, there will be different approaches to recommender systems, which are explained below:
Multi-criteria recommender systems: Various conditions like age, gender, location, likes, and dislikes are used for categorization and then items are suggested. E.g., suggestion of apparel based on age and gender.
Risk-aware recommender systems: There is always uncertainty when users use Internet applications (website or mobile). Recommending any advertisement over the Internet must consider risk and users must be aware of this. E.g., advertisement display suggestion over Internet application.
Mobile recommender systems: These are location-based suggestions that consist of users’ current location or future location and provide suggestions based on that. E.g., mostly preferred in traveling and tourism.
Hybrid recommender systems: These are the combination of multiple approaches for recommendations. E.g., suggestion of hotels and restaurants based on user preference and travel information.
Collaborative and content recommender systems: These are the combination of collaborative and content-based approaches. E.g., suggestion of the highest-rated movie of users’ preference along with their watch history.
Practical Example with Implementation
In this example, we have a sample dataset of drugs prescribed for various medical conditions and ratings given by patients. What we need here is for any medical condition we have to receive a suggestion for the most suitable prescribed drugs for treatment.
Sample Dataset:
Below is the sample of the publicly available medical drug dataset used from the Winter 2018 Kaggle University Club Hackathon.
drugName
condition
rating
condition_id
Mirtazapine
Depression
10
201
Mesalamine
Crohn’s Disease, Maintenance
8
185
Bactrim
Urinary Tract Infection
9
657
Contrave
Weight Loss
9
677
Cyclafem 1 / 35
Birth Control
9
122
Zyclara
Keratosis
4
365
Copper
Birth Control
6
122
Amitriptyline
Migraine Prevention
9
403
Methadone
Opiate Withdrawal
7
460
Levora
Birth Control
2
122
Paroxetine
Hot Flashes
1
310
Miconazole
Vaginal Yeast Infection
6
664
Belviq
Weight Loss
1
677
Seroquel
Schizoaffective Disorde
10
575
Ambien
Insomnia
2
347
Nuvigil
Narcolepsy
9
424
Chantix
Smoking Cessation
10
597
Microgestin Fe 1 / 20
Acne
3
49
Klonopin
Bipolar Disorde
6
121
Ciprofloxacin
Urinary Tract Infection
10
657
Trazodone
Insomnia
1
347
EnteraGam
Irritable Bowel Syndrome
9
356
Aripiprazole
Bipolar Disorde
1
121
Cyclosporine
Keratoconjunctivitis Sicca
1
364
Sample Code:
We will do this in 5 steps:
1. Importing required libraries
2. Reading the drugsComTest_raw.csv file and creating a pivot matrix.
3. Creating a KNN model using the NearestNeighbors function with distance metric- ‘cosine’ & algorithm- ‘brute’. Possible values for distance metric are ‘cityblock’, ‘euclidean’, ‘l1’, ‘l2’ & ‘manhattan’. Possible values for the algorithm are ‘auto’, ‘ball_tree’, ‘kd_tree’, ‘brute’ & ‘cuml’.
4. Selecting one medical condition randomly for which we have to suggest 5 drugs for treatment.
5. Finding the 6 nearest neighbors for the sample, calling the kneighbors function with the trained KNN models created in step 3. The first k-neighbor for the sample medical condition is self with a distance of 0. The next 5 k-neighbors are drugs prescribed for the sample medical condition.
This is the collaborative-based recommendation system that uses the patients’ ratings of given drug treatments to find similarities in medical conditions. Here, we are matching the patterns for ratings given to drugs by patients. This system compares all the rating patterns and tries to find similarities (cosine similarity).
Challenges of Recommendation System
Any recommendation system requires a decent quantity of quality information to process. Before developing such a system, we must be aware of it. Acknowledging and handling such challenges improve the accuracy of recommendation.
1. Cold Start: Recommending a new user or a user without any previous behavior is a problem. We can recommend the most popular options to them. E.g., YouTube videos suggestion for newly registered users.
2. Not Enough Data: Having insufficient data provides recommendations with less certainty. E.g., suggestion of hotels or restaurants will not be accurate if systems are uncertain about users’ locations.
3. Grey Sheep Problem: This problem occurs when the inconsistent behavior of a user makes it difficult to find a pattern. E.g., multiple users are using the same account, so user activity will be wide, and the system will have difficulty in mapping such patterns.
4. Similar items: In these cases, there is not enough data to separate similar items. For these situations, we can recommend all similar items randomly. E.g., apparel suggestions for users with color and sizes. All shirts are similar.
5. Shilling Attacks: Intentional negative behavior that leads to bad/unwanted recommendations. While immoral, we cannot deny the possibility of such attacks. E.g., user ratings and reviews over various social media platforms.
Accuracy and Performance Measures
Accuracy evaluation is important as we always follow and try to improve algorithms. The most preferred measures for improving algorithms are user studies, online evaluations, and offline evaluations. Our recommendation models must be ready to learn from users’ activity daily. For online evaluations, we have to regularly test our recommendation system.
If we understand the challenges of the recommendation system, we can prepare such testing datasets to test its accuracy. With these variations of datasets, we can improve our approach of user studies and offline evaluations.
1. Online Evaluations: In online evaluations, prediction models are updated frequently with the unmonitored data, which leads to the possibility of unexpected accuracy. To verify this, the prediction models are exposed to the unmonitored data with less uncertainty and then the uncertainty of unmonitored data is gradually increased.
2. Offline Evaluations: In offline evaluations, the prediction models are trained with a sample dataset that consists of all possible uncertainty with expected outcomes. To verify this, the sample dataset will be gradually updated and prediction models will be verified with predicted and actual outcomes. E.g., creating multiple users with certain activity and expecting genuine suggestions for them.
Conclusion
As a part of this article, we have learned about the approaches, challenges, and evaluation methods, and then we created a practical example of the collaboration-based recommendation system. We also explored various types and filtering approaches with real-world scenarios.
We have also executed sample code with a publicly available medical drug dataset with patient ratings. We can opt for various options for distance matrix and algorithm for the NearestNeighbors calculation. We have also listed various challenges for this system and understood the accuracy evaluation measures and things that affect and improve them.
According to the OpenAI Gym GitHub repository “OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. This is the gym open-source library, which gives you access to a standardized set of environments.”
Open AI Gym has an environment-agent arrangement. It simply means Gym gives you access to an “agent” which can perform specific actions in an “environment”. In return, it gets the observation and reward as a consequence of performing a particular action in the environment.
There are four values that are returned by the environment for every “step” taken by the agent.
Observation (object): an environment-specific object representing your observation of the environment. For example, board state in a board game etc
Reward (float): the amount of reward/score achieved by the previous action. The scale varies between environments, but the goal is always to increase your total reward/score.
Done (boolean): whether it’s time to reset the environment again. E.g you lost your last life in the game.
Info (dict): diagnostic information useful for debugging. However, official evaluations of your agent are not allowed to use this for learning.
Following are the available Environments in the Gym:
Classic control and toy text
Algorithmic
Atari
2D and 3D robots
Here you can find a full list of environments.
Cart-Pole Problem
Here we will try to write a solve a classic control problem from Reinforcement Learning literature, “The Cart-pole Problem”.
The Cart-pole problem is defined as follows: “A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center.”
The following code will quickly allow you see how the problem looks like on your computer.
import gymenv = gym.make('CartPole-v0')env.reset()for _ in range(1000): env.render() env.step(env.action_space.sample())
This is what the output will look like:
Coding the neural network
#We first import the necessary libraries and define hyperparameters - import gymimport randomimport numpy as npimport tflearnfrom tflearn.layers.core import input_data, dropout, fully_connectedfrom tflearn.layers.estimator import regressionfrom statistics import median, meanfrom collections import CounterLR = 2.33e-4env = gym.make("CartPole-v0")observation = env.reset()goal_steps = 500score_requirement = 50initial_games = 10000#Now we will define a function to generate training data - def initial_population(): # [OBS, MOVES] training_data = [] # all scores: scores = [] # scores above our threshold: accepted_scores = [] # number of episodes for _ in range(initial_games): score = 0 # moves specifically from this episode: episode_memory = [] # previous observation that we saw prev_observation = [] for _ inrange(goal_steps): # choose random action left or right i.e (0 or 1) action = random.randrange(0,2) observation, reward, done, info = env.step(action) # since that the observation is returned FROM the action # we store previous observation and corresponding actioniflen(prev_observation) >0 : episode_memory.append([prev_observation, action]) prev_observation = observation score+=rewardifdone: break # reinforcement methodology here. # IF our score is higher than our threshold, we save # all we're doing is reinforcing the score, we're not trying # to influence the machine in any way astoHOWthatscoreis # reached.if score >=score_requirement: accepted_scores.append(score) for data inepisode_memory: # convert to one-hot (this is the output layer for our neural network)if data[1] ==1: output = [0,1] elif data[1] ==0: output = [1,0] # saving our training data training_data.append([data[0], output]) # reset env to play again env.reset() # save overall scores scores.append(score)# Now usingtflearn we will define our neural network def neural_network_model(input_size): network =input_data(shape=[None, input_size, 1], name='input') network =fully_connected(network, 128, activation='relu') network =dropout(network, 0.8) network =fully_connected(network, 256, activation='relu') network =dropout(network, 0.8) network =fully_connected(network, 512, activation='relu') network =dropout(network, 0.8) network =fully_connected(network, 256, activation='relu') network =dropout(network, 0.8) network =fully_connected(network, 128, activation='relu') network =dropout(network, 0.8) network =fully_connected(network, 2, activation='softmax') network =regression(network, optimizer='adam', learning_rate=LR, loss='categorical_crossentropy', name='targets') model = tflearn.DNN(network, tensorboard_dir='log')return model#It is time to train the model now -def train_model(training_data, model=False):X= np.array([i[0] for i in training_data]).reshape(-1,len(training_data[0][0]),1) y = [i[1] for i in training_data]if not model: model =neural_network_model(input_size =len(X[0])) model.fit({'input': X}, {'targets': y}, n_epoch=5, snapshot_step=500, show_metric=True, run_id='openai_CartPole')return modeltraining_data =initial_population()model =train_model(training_data)#Training complete, now we should play the game to see how the output looks like scores = []choices = []for each_game inrange(10): score =0 game_memory = [] prev_obs = [] env.reset() for _ inrange(goal_steps): env.render()iflen(prev_obs)==0: action = random.randrange(0,2)else: action = np.argmax(model.predict(prev_obs.reshape(-1,len(prev_obs),1))[0]) choices.append(action) new_observation, reward, done, info = env.step(action) prev_obs = new_observation game_memory.append([new_observation, action]) score+=rewardifdone: break scores.append(score)print('Average Score:',sum(scores)/len(scores))print('choice 1:{} choice 0:{}'.format(float((choices.count(1))/float(len(choices)))*100,float((choices.count(0))/float(len(choices)))*100))print(score_requirement)
This is what the result will look like:
Conclusion
Though we haven’t used the Reinforcement Learning model in this blog, the normal fully connected neural network gave us a satisfactory accuracy of 60%. We used tflearn, which is a higher level API on top of Tensorflow for speeding-up experimentation. We hope that this blog will give you a head start in using OpenAI Gym.
We are waiting to see exciting implementations using Gym and Reinforcement Learning. Happy Coding!