micro/services

Real World Micro Services

Micro Services

Real world Micro services

Overview

Micro services provide the fundamental building blocks for any products, apps or services. They can be used in isolation or combined to create powerful distributed systems. The services are intended to be consumed by each other using RPC and externally through the Micro API.

Rationale

Read more about the reason for Micro Services in this blog post.

Interfaces

Every service starts with a protobuf interface definition, which is a standard used by Google and everyone else now that gRPC is so dominant. The idea is to define the API in protobuf, code generate and implement the handlers for it. The services can be called by other services on the platform using those code generated clients or an API Gateway, which Micro provides. External calls via the API use the same format but with HTTP/JSON endpoints.

Services

Services available thus far:

• address - Address lookup by postcode
• analytics - Track and retrieve events
• answer - Instant answers to any question
• app - Serverless app deployment
• avatar - Generate an avatar
• bitcoin - Bitcoin price and transaction info
• cache - Fast access key-value storage
• carbon - Purchase carbon offsets
• chat - Instant messaging service
• comments - Add comments to any App
• contact - Store your contacts
• cron - Schedule cron jobs
• crypto - Cryptocurrency prices, quotes, and news
• currency - Exchange rates and currency conversion
• db - Serverless postgres database
• dns - DNS over HTTPS (DoH)
• email - Send emails in a flash
• emoji - All the emojis you need 🎉
• ethereum - Ethereum API explorer
• evchargers - Find electric vehicle (EV) chargers wherever you go
• event - Event stream processing
• file - Store, list, and retrieve text files
• forex - Foreign exchange (FX) rates
• function - Serverless lambda functions
• geocoding - Address geocoding and reverse lookup
• gifs - Quick and simple GIF search
• google - Google search service
• helloworld - Just saying hello world
• holidays - Find the holidays observed in a particular country
• id - Generate unique IDs (uuid, snowflake, etc)
• image - Upload, resize, and convert images
• ip - IP to geolocation lookup
• joke - Funny Jokes
• lists - Make a list
• location - Real time GPS location tracking and search
• memegen - Generate funny memes
• minecraft - Minecraft server ping
• movie - Search for movies
• mq - PubSub messaging
• news - Get the latest news
• nft - Explore NFT Assets
• notes - Store and retrieve notes
• otp - One time password generation
• password - Generate strong passwords
• ping - Ping any IP
• place - Search for places
• postcode - Fast UK postcode lookup
• prayer - Islamic prayer times
• price - Global commodities index
• qr - QR code generator
• quran - The Holy Quran
• routing - Etas, routes and turn by turn directions
• rss - RSS feed crawler and reader
• search - Indexing and full text search
• sentiment - Real time sentiment analysis
• sms - Send SMS messages
• space - Infinite cloud storage
• spam - Check if an email is spam
• stock - Live stock quotes and prices
• stream - Ephemeral message streams
• sunnah - Traditions and practices of the Islamic prophet, Muhammad (pbuh)
• thumbnail - Create website thumbnails
• time - Time, date, and timezone info
• translate - Language translation service
• tunnel - Tunnel HTTP requests
• twitter - Realtime twitter timeline & search
• url - URL shortening, sharing, and tracking
• user - Authenticate and manage users
• vehicle - UK vehicle lookup
• wallet - Virtual Wallet
• weather - Real time weather forecast
• wordle - Multiplayer wordle
• youtube - Search for YouTube videos

Usage

Micro Services depend on Micro

Run Micro

Install and run the server first

micro server

Run a Service

Run a service from source

micro run github.com/micro/services/helloworld

Call a Service

To call a service from another

import "github.com/micro/services/helloworld/proto"

Call it through the API

curl "http://localhost:8080/helloworld/Call?name=Alice"

From the command line

micro helloworld call --name=Alice

Browse to

http://localhost:8082/helloworld/Call

Contribute

We welcome contributions of additional services:

• Services must be built using the Micro platform
• Any dependency must be configured using the Micro Config
• All services must include a README.md and be well commented

Config

Config for each service should be stored under the service key by name

E.g db postgres host address db.address

services's Issues

App API

Provide a simple container API for running containers. Doesn't have to be on our own infra. Can be google cloud run.

Place API

Create a place API powered by google places. This would allow you to search for places based on location, nearby and do useful things like autocomplete in a UI.

Todo: replace sendgrid-go with Email Service in User service

User service uses package github.com/sendgrid/sendgrid-go to send email.
But there is Email Service which can replace sendgrid-go;

services/user/domain/domain.go

Lines 81 to 98 in ff49510

	func (domain *Domain) SendEmail(fromName, toAddress, toUsername, subject, textContent, token, redirctUrl, failureRedirectUrl string) error {
	if domain.sengridKey == "" {
	return fmt.Errorf("empty email api key")
	}
	from := mail.NewEmail(fromName, domain.fromEmail)
	to := mail.NewEmail(toUsername, toAddress)
	// set the text content
	textContent = strings.Replace(textContent, "$micro_verification_link", "https://user.m3o.com?token="+token+"&redirectUrl="+url.QueryEscape(redirctUrl)+"&failureRedirectUrl="+url.QueryEscape(failureRedirectUrl), -1)
	message := mail.NewSingleEmail(from, subject, to, textContent, "")
	// send the email
	client := sendgrid.NewSendClient(domain.sengridKey)
	response, err := client.Send(message)
	logger.Info(response)
	return err
	}

TODO: Replace DB with Store

Replaced DB usage with Store in the following services

• Function
• User

CDN API

CDN api built on s3 serving cdn.m3ocontent.com

Movie API

Movie appears to be top searched along with Weather.

Create an API based on https://www.themoviedb.org/documentation/api or https://developer.imdb.com/

http://www.omdbapi.com/

Ping API

You give a url of the website and micro pings it. (to see if the site is online)

Data API (Powered by MongoDB)

Build a Data API powered by MongoDB Atlas https://www.mongodb.com/cloud/atlas

Rewards API

Earn rewards for engagement

Meme API

Bringing memes to the Micro platform

Answer API: example question gives different response now

I tried calling the answer service for questions with the query "google" as shown in the example in this repo services/answer/examples.json.

It does not seem to produce the shown response for the query anymore, instead giving the message Don't have an answer for that but here's a related topic: and then appending the same google description text as before. See below for the real response.

{
  "answer": "Don't have an answer for that but here's a related topic: Google An American multinational technology company that specializes in Internet-related services...",
  "url": "https://duckduckgo.com/Google",
  "image": "https://duckduckgo.com/i/8f85c93f.png"
}

Seems like duckduckgo changed something about this specific keyword, since other companies like microsoft work exactly as described. Might make sense to switch to another example keyword, since this one is not applicable anymore.

TODO: Replace model with store

Replace the usage of model with store in the following services

• URL
• RSS
• File

Avatar API

Generate user avatars

Ethereum API

Browse the ethereum blockchain

Donate API

Enable donations via api

• $1
• $5
• $10

WhatsApp API

https://www.reddit.com/r/api/comments/yxeqk6/api_request_looking_for_whatsapp_status_tracker/

Bitcoin API

Browse the bitcoin blockchain

TODO: Add mailing lists to email API

Add the ability to manage mailing lists to the email service.

• Email.List - List all the emails in a list
• Email.Subscribe - Subscribe to a list

Function API

Wrap GCF or related to provide functions api

Cron API

Cron API which lets things be executed on a schedule and calls an endpoint

Presence API

Required to determine whether someone is present

exec: "go": executable file not found in $PATH

Hello.
I followed https://micro.mu/getting-started#getting-started upto micro run github.com/micro/services/helloworld and got "go": executable file not found in $PATH error message.
~# ls -l which go
-rwxr-xr-x 1 root root 14244478 Dec 3 17:35 /usr/local/go/bin/go
~# micro run github.com/micro/services/helloworld
exec: "go": executable file not found in $PATH
But go apparently is in $PATH. What could be the problem?

Thanks

OpenAPI generator turns proto maps to openapi arrays

See

map<string,string> metadata = 9;

                "metadata": {
                  "items": {
                    "properties": {
                      "key": {
                        "type": "string"
                      },
                      "value": {
                        "type": "string"
                      }
                    },
                    "type": "object"
                  },
                  "type": "array"
                },

TODO: Image service

We need an image service with the following endpoints

• Upload - allocates and returns an s3 bucket where an image can be uploaded/served from
• Convert - takes an image and converts formats from jpg to png or whatever else
• Resize - can resize an image to shrink the scale
• Snapshot - Takes a website url and snapshots a thumbnail

File service bug

{"Id":"go.micro.client","Code":500,"Detail":"grpc: failed to unmarshal the received message json: cannot unmarshal array into Go value of type map[string]json.RawMessage","Status":"Internal Server Error"

NFT API

https://docs.opensea.io/

Discord API

micro services is giving connection closed error

I am trying to run hellowrld example by following steps as per
https://micro.mu/getting-started
I started micro server

I can see server started, however when I try to login or list services it is giving connection closed error.

Please help me in if I am missing anything.

Thanks,
Sreenatha

Metrics (analytics) API

Provide a simple metrics/analytics API that lets you track counters and related things to do analytics.

Charity API

Donate to charity with endpoints 1/5/10/20 which equate to dollars donated.

Use a predefined charity API to build on.

DB read limit / offset / order

Add limit / offset and order to the DB Read method. Set max limit to 1000 and return an error if the user specifies over it. Set default limit to 25 if no limit is specified.

Chat API

Resurface https://github.com/micro/services/tree/master/test/chat

Translate API

Language translation api

TODO: Timezone API

Leverage free timezone dataset to provide an api for timezones

https://timezonedb.com/

Reminder API

An API to provide scheduled reminders. Simple text based reminders to start. Eventually when in promixity to something (location based)

Add Raw HTML screenshot support

Support the ability to pass raw html to the thumbnail service to screenshot

TODO: QR Code API

Jokes API

Leverage https://github.com/taivop/joke-dataset for a jokes api

Tube API

London underground tube api

test/pubsub is not receiving any messages!

Followed the readme https://github.com/micro/services/tree/master/test/pubsub and got output:

micro logs pubsub
2021-02-04 02:37:58  file=service/service.go:195 level=info Starting [service] pubsub
2021-02-04 02:37:58  file=grpc/grpc.go:939 level=info Server [grpc] Listening on [::]:46639
2021-02-04 02:37:58  file=grpc/grpc.go:956 level=info Broker [service] Connected to :8003
2021-02-04 02:37:58  file=grpc/grpc.go:769 level=info Registry [service] Registering node: pubsub-34d386f6-2871-4a80-a380-66406da9e353
2021-02-04 02:37:58  file=grpc/grpc.go:798 level=info Subscribing to topic: messages

micro version v3.0.4

Support Content-Type in image service

Use the existing content-type from an image url for the image itself. Currently becomes blob type as default defined by the s3 plugin.

Redis API

You get your own redis cache instance! (with a memory limit)

Search API

Build on Algolia

Whiteboard API

TODO: Standardise config access

Config keys are all over the place. It needs to be a standard model of naming.

Subscribe API

Provide a simple API to manage mailing lists

How to use the *.mu file?

How to use the *.mu file?

Contact API

Store your contacts

Space API

Create a space API that provides object storage and potentially CDN URLs for data

Basically blob storage.

Carbon API

Carbon offsetting API