vanus-labs/vanus-connect

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors.

Vanus Connect

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling you to have faster integrations with no codes. All data produced or processed by our connectors conforms to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

vanus-connect's Issues

[Good First Issues]: review Vanus Connect Documentation

Motivation

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors.

Reviewing Vanus connector docs will be a good start if you decide to know more about Vanus Connect.

Task List

• #133
• #135
• #136
• #137

Contribution points

Each issue counts specific points based on its difficulty. Contributors earn the corresponding points if they solve that issue. Contribution points are used to describe the contributions contributors have made. They can also be used to get rewards when there are community events.

How to claim to solve the issue

If you want to solve a specific issue, please leave a comment in that issue like:

I'd like to solve this issue, please assign it to me.

Vanus community will assign the issue to you on time.

test lqp

this is test

refactor MongoDB Sink with latest cdk

refactor MongoDB Sink with latest cdk

[Good First Issues]: update connector doc "connectors/sink-slack" - 1 point

Motivation

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors.

Reviewing Vanus connector docs will be a good start if you decide to know more about Vanus Connect.

Details

Update the improper image sources in the documentation of sink-slack.

How to change

1. Fork the vanus-connect repository.
2. Create a branch update-sink-slack
3. Update file connectors/sink-slack/README.md
4. Change image source https://github.com/linkall-labs/vance/blob/main/connectors/sink-slack/message.png to https://github.com/linkall-labs/vance/blob/main/connectors/sink-slack/message.png?raw=true.
5. Submit your PR

Contribution points

Each issue counts specific points based on its difficulty. Contributors earn the corresponding points if they solve that issue. Contribution points are used to describe the contributions contributors have made. They can also be used to get rewards when there are community events.

This issue counts 1 point.

How to claim to solve this issue

If you want to solve this issue, please leave a comment in this issue like:

I'd like to solve the issue, please assign it to me.

Vanus community will assign the issue to you on time.

test

test body

🎯New Source Connector Request: test Source Connector

Source Connector Twitter

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents and sends it to the target.

• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents, and send them to the target.

• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.

• A push connector is passive and only does its logic when it receives an event
• A pull connector must pull the data in an interval. We use a push method when it’s possible

Features of this Source Connector

Receive events from TikTok when a user’s post is liked, shared, or commented on, and when a user is followed and put the data into a CloudEvent to send it to the target

Specific data needed

When someone likes the user's post

• Post ID: The unique identifier for the post that was liked. The post could be a video.
• User ID: The unique identifier for the user who liked the post.
• User screen name: The screen name (i.e., username) of the user who liked the post.
• Like creation time: The timestamp indicates when the like was created.
• Post URL: The URL of the post that was liked.
• Number of likes: The total number of likes the post has received.

When a post is commented on

• Post ID: The unique identifier for the post that was commented on. The post could be a video.
• User ID: The unique identifier for the user who commented on the post.
• User screen name: The screen name (i.e., username) of the user who commented on the post.
• Number of Comments: The total number of replies the post has received.
• Post URL: The URL of the post that was commented on.
• Comment creation time: The timestamp indicates when the comment was created.

When a user is followed

• User ID: The unique identifier for the user who followed.
• Number of Followers: The total Number of Followers the user.
• Follow creation time: The timestamp indicates when the user was followed.

How to create a new connector

• Fork the vanus-connect repo.
• Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
• Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
• Implement your connector and submit a PR.
• Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

🎯New Source Connector Request: TikTok Source Connector

Source Connector Twitter

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents and sends it to the target.

• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents, and send them to the target.

• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.

• A push connector is passive and only does its logic when it receives an event
• A pull connector must pull the data in an interval. We use a push method when it’s possible

Features of this Source Connector

Receive events from TikTok when a user’s post is liked, shared, or commented on, and when a user is followed and put the data into a CloudEvent to send it to the target

Specific data needed

When someone likes the user's post

• Post ID: The unique identifier for the post that was liked. The post could be a video.
• User ID: The unique identifier for the user who liked the post.
• User screen name: The screen name (i.e., username) of the user who liked the post.
• Like creation time: The timestamp indicates when the like was created.
• Post URL: The URL of the post that was liked.
• Number of likes: The total number of likes the post has received.

When a post is commented on

• Post ID: The unique identifier for the post that was commented on. The post could be a video.
• User ID: The unique identifier for the user who commented on the post.
• User screen name: The screen name (i.e., username) of the user who commented on the post.
• Number of Comments: The total number of replies the post has received.
• Post URL: The URL of the post that was commented on.
• Comment creation time: The timestamp indicates when the comment was created.

When a user is followed

• User ID: The unique identifier for the user who followed.
• Number of Followers: The total Number of Followers the user.
• Follow creation time: The timestamp indicates when the user was followed.

How to create a new connector

• Fork the vanus-connect repo.
• Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
• Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
• Implement your connector and submit a PR.
• Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

🎯New Source Connector Request: Facebook Marketing Source Connector

Source Connector Facebook Marketing

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents, and sends it to the target.

• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents, and send them to the target.

• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.

• A push connector is passive and only does its logic when it receives an event
• A pull connector must pull the data in an interval. We use a push method when it’s possible

Features of this Source Connector

Receive events from Facebook Marketing API for Ad Campaigns, Ads

Specific data needed

Ads Campaign

• Campaign ID: A unique identifier for the Ad Campaign.
• Campaign Name: The name of the Ad Campaign.
• Campaign Objective: The objective of the Ad Campaign, such as brand awareness, lead generation, or website traffic.
• Campaign Status: The current status of the Ad Campaign, such as active, paused, or deleted.
• Campaign Delivery Insights: Insights into the delivery and performance of the Ad Campaign, including metrics like reach, impressions, frequency, and cost per result.
• Campaign Spend: The amount of money spent on the Ad Campaign.
• Campaign Budget: The budget allocated to the Ad Campaign.
• Campaign Schedule: The start and end dates of the Ad Campaign.
• Ad Account ID: The ID of the Ad Account associated with the Ad Campaign.
• Campaign Objective Details: Additional details about the Ad Campaign objective, such as the specific action the campaign is optimized for, such as app installs or page likes.
• Campaign Insights: Detailed insights into the performance of the Ad Campaign, including metrics like click-through rate, conversion rate, and cost per conversion.

Ads

• Ad ID: A unique identifier for the Ad.
• Ad Name: The name of the Ad.
• Ad Creative: The content and design of the Ad, including images, videos, and text.
• Ad Format: The format of the Ad, such as single image, carousel, or video.
• Ad Placement: The placement of the Ad, such as News Feed, Instagram Stories, or Audience Network.
• Ad Delivery Insights: Insights into the delivery and performance of the Ad, including metrics like reach, impressions, frequency, and cost per result.
• Ad Spend: The amount of money spent on the Ad.
• Ad Schedule: The start and end dates of the Ad.
• Ad Status: The current status of the Ad, such as active, paused, or deleted.
• Ad Creative Insights: Detailed insights into the performance of the Ad creative, including metrics like click-through rate, engagement rate, and conversion rate.
• Ad Targeting: The targeting criteria used for the Ad, including demographic, geographic, and interest-based targeting.
• Ad Performance Metrics: Performance metrics for the Ad, including metrics like impressions, clicks, conversions, and cost per result.

How to create a new connector

• Fork the vanus-connect repo.
• Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
• Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
• Implement your connector and submit a PR.
• Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

This issue is a test

test 1

this is just a test

Test

refactor display sink

• refactor by Golang
• support text/plain and application/json

wrong official image source

the image source is

https://hub.docker.com/r/vancehub/source-mysql

test

this is a test

test

test

[Good First Issues]: Review Elasticsearch Sink

Documentation Review

Motivation

The best way to learn about an open-source project starts with reading its documentation. In order to better know about Vanus and improve the quality of Vanus documentation, we're looking for OSS enthusiasts to review our docs.

Details

Review and test connectors documentation

Requirements

When you test:

• Use the playground to test GitHub Source
• Run the connector according to the documentation
• Test every link in the documentation

When you review the documentation:

• Grammar check
• Wrong capitalization
• All image are working
• You can understand every step.

If you have any questions or comments, please feel free to leave a comment in this issue.

How to change it:

Comment to Claim this issue.

1. Fork the docs repository.
2. Create a branch Review GitHub Source .
3. Review document Elasticsearch Sink
4. Change: Use the playground terminal to test the connector, Review document.
5. Make a PR.

🎯New Source Connector Request: Google Analytics Source Connector

Source Connector Google Analytics

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents, and sends it to the target.

• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents, and send them to the target.

• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.

• A push connector is passive and only does its logic when it receives an event
• A pull connector must pull the data in an interval. We use a push method when it’s possible

Features of this Source Connector

Receive events from Google Analytics for Audience Demographics, Traffic Sources, User Behavior, Content Performance.

Specific data needed

Audience Demographics

• Location: The geographic location of website visitors, including country, region, and city.
• Language: The language preferences of website visitors.
• Technology: The technology used by website visitors to access the site, including browser, operating system, and device type.

Traffic Sources

• Organic Search: The number of website visitors who arrived at the site through organic search results, along with the keywords used to find the site.
• Paid Search: The number of website visitors who arrived at the site through paid search campaigns, along with information about the campaigns, keywords, and ad groups.
• Direct Traffic: The number of website visitors who arrived at the site by typing the URL directly into the browser or using a bookmark.
• Referral Traffic: The number of website visitors who arrived at the site through links on other websites, along with information about the referring site.
• Social Media: The number of website visitors who arrived at the site through social media channels, including the specific social media platform and the content that brought them to the site.
• Email: The number of website visitors who arrived at the site through email campaigns, along with information about the campaigns, open rates, and click-through rates.
• Display Advertising: The number of website visitors who arrived at the site through display advertising campaigns, along with information about the campaigns, impressions, and click-through rates.
• Affiliates: The number of website visitors who arrived at the site through affiliate marketing campaigns, along with information about the affiliates and their promotional activities.

User Behaviour

• Pageviews: The number of times each page on the website has been viewed by visitors.
• Time on Page: The average amount of time visitors spend on each page.
• Bounce Rate: The percentage of visitors who leave the website after viewing only one page.
• Site Search: Information about how visitors are using the site search function, including the terms they search for and the pages they visit as a result.
• Events: Information about specific user interactions with the website, such as clicks on buttons or links, video views, and form submissions.

Content Performance

• Pageviews: The number of times each page on the website has been viewed by visitors.
• Unique Pageviews: The number of individual visitors who viewed each page.
• Time on Page: The average amount of time visitors spend on each page.
• Bounce Rate: The percentage of visitors who leave the website after viewing only one page.
• Exit Pages: The pages on the website where visitors are most likely to leave.
• Site Speed: Information about the loading speed of individual pages on the website, including average page load time and suggestions for improvement.

How to create a new connector

• Fork the vanus-connect repo.
• Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
• Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
• Implement your connector and submit a PR.
• Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus Community will assign the issue to you on time.

standardized Connector Title

'sink-tencentcloud-scf' is more clarity

🎯 New Sink Connector Request: Google Sheets

Sink Connector-Google Sheets

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conforms to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Sink Connector

A sink connector receives CloudEvents and does some specific business logics. For example:

• MySQL Sink extracts useful data from CloudEvents and writes them to a MySQL database.
• Display Sink prints incoming CloudEvents out.
• Slack Sink extracts data from CloudEvents and send them to a Slack channel.

Features of this Connector

• Sink-Google-Sheets allows users to insert a new row based on the incoming CloudEvents.

How to create a new connector

1. Fork the vanus-connect repo.
2. Create a sink_google_sheets_proposal.md based on the sink_proposal_example.md under vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
3. Copy one of the sink templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as sink-google-sheets.
4. Implement your connector and submit a PR.
5. Wait the connector to be reviewed and approved.

How to claim to implement this connector

f you want to solve this issue, please leave a comment in this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

🎯New Source Connector Request: test Source Connector

Source Connector Twitter

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents and sends it to the target.

• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents, and send them to the target.

• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.

• A push connector is passive and only does its logic when it receives an event
• A pull connector must pull the data in an interval. We use a push method when it’s possible

Features of this Source Connector

Receive events from TikTok when a user’s post is liked, shared, or commented on, and when a user is followed and put the data into a CloudEvent to send it to the target

Specific data needed

When someone likes the user's post

• Post ID: The unique identifier for the post that was liked. The post could be a video.
• User ID: The unique identifier for the user who liked the post.
• User screen name: The screen name (i.e., username) of the user who liked the post.
• Like creation time: The timestamp indicates when the like was created.
• Post URL: The URL of the post that was liked.
• Number of likes: The total number of likes the post has received.

When a post is commented on

• Post ID: The unique identifier for the post that was commented on. The post could be a video.
• User ID: The unique identifier for the user who commented on the post.
• User screen name: The screen name (i.e., username) of the user who commented on the post.
• Number of Comments: The total number of replies the post has received.
• Post URL: The URL of the post that was commented on.
• Comment creation time: The timestamp indicates when the comment was created.

When a user is followed

• User ID: The unique identifier for the user who followed.
• Number of Followers: The total Number of Followers the user.
• Follow creation time: The timestamp indicates when the user was followed.

How to create a new connector

• Fork the vanus-connect repo.
• Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
• Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
• Implement your connector and submit a PR.
• Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

test

test body

[TEST]: 🎯 New Sink Connector Request: MySQL

Sink MySQL

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conforms to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Sink Connector

A sink connector receives CloudEvents and does some specific business logics. For example:

• MySQL Sink extracts useful data from CloudEvents and writes them to a MySQL database.
• Display Sink prints incoming CloudEvents out.
• Slack Sink extracts data from CloudEvents and send them to a Slack channel.

Features of this Connector

• Sink-MySQL allows users to insert a new row on MySQL tables based on the incoming CloudEvents.

How to create a new connector

1. Fork the vanus-connect repo.
2. Create a sink_mysql_proposal.md based on the sink_proposal_example.md under vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
3. Copy one of the sink templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as sink-mysql.
4. Implement your connector and submit a PR.
5. Wait the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment in this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

[Good First Issues]: update connector doc "connectors/sink-feishu" - 2 points

Motivation

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors.

Reviewing Vanus connector docs will be a good start if you decide to know more about Vanus Connect.

Details

Update the improper image sources in the documentation of sink-feishu.

How to change

1. Fork the vanus-connect repository.
2. Create a branch update-sink-feishu
3. Update file connectors/sink-feishu/README.md
4. Change image source https://github.com/linkall-labs/vance-docs/raw/main/resources/connectors/sink-feishu-bot/add-a-bot.gif to https://github.com/linkall-labs/vance/blob/main/connectors/sink-feishu/add-a-bot.gif?raw=true.
5. Change image source https://github.com/linkall-labs/vance-docs/raw/main/resources/connectors/sink-feishu-bot/feishu-config.png to https://github.com/linkall-labs/vance/blob/main/connectors/sink-feishu/feishu-config.png?raw=true.
6. Change image source https://github.com/linkall-labs/vance/blob/main/connectors/sink-feishu/received-message.png to https://github.com/linkall-labs/vance/blob/main/connectors/sink-feishu/received-message.png?raw=true.
7. Submit your PR

Contribution points

Each issue counts specific points based on its difficulty. Contributors earn the corresponding points if they solve that issue. Contribution points are used to describe the contributions contributors have made. They can also be used to get rewards when there are community events.

This issue counts 2 points.

How to claim to solve this issue

If you want to solve this issue, please leave a comment in this issue like:

I'd like to solve the issue, please assign it to me.

Vanus community will assign the issue to you on time.

HTTP sink and Source need update with updated CDK

[Good First Issues]: Review Slack Sink

Documentation Review

Motivation

The best way to learn about an open-source project starts with reading its documentation. In order to better know about Vanus and improve the quality of Vanus documentation, we're looking for OSS enthusiasts to review our docs.

Details

Review and test connectors documentation

Requirements

When you test:

• Use the playground to test it
• Run the connector according to the documentation
• Test every link in the documentation

When you review the documentation:

• Grammar check
• Wrong capitalization
• All image are working
• You can understand every step.

If you have any questions or comments, please feel free to leave a comment in this issue.

How to change it:

Comment to Claim this issue.

1. Fork the docs repository.
2. Create a branch
3. Review document Slack Sink
4. Change: Use the playground terminal to test the connector, Review document.
5. Make a PR.

test

test body

test

test

test body

[Good First Issues]: Review Amazon SQS Source

Documentation Review

Motivation

The best way to learn about an open-source project starts with reading its documentation. In order to better know about Vanus and improve the quality of Vanus documentation, we're looking for OSS enthusiasts to review our docs.

Details

Review and test connectors documentation

Requirements

When you test:

• Use the playground to test the connector
• Run the connector according to the documentation
• Test every link in the documentation

When you review the documentation:

• Grammar check
• Wrong capitalization
• All image are working
• You can understand every step.

If you have any questions or comments, please feel free to leave a comment on this issue.

How to change it:

Comment to Claim this issue.

1. Fork the docs repository.
2. Create a branch Review-Amazon-sqs .
3. Review document Amazon SQS Source
4. Change: Use the playground terminal to test the connector, Review document.
5. Make a PR.

🎯 New Sink Connector Request: Facebook Lead Ads

Sink Connector-Facebook-Lead-Ads

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conforms to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Sink Connector

A sink connector receives CloudEvents and does some specific business logics. For example:

• MySQL Sink extracts useful data from CloudEvents and writes them to a MySQL database.
• Display Sink prints incoming CloudEvents out.
• Slack Sink extracts data from CloudEvents and send them to a Slack channel.

Features of this Connector

• Create a lead form: Create a lead form for ad campaigns, specifying the questions and fields you want to collect from users.

How to create a new connector

1. Fork the vanus-connect repo.
2. Create a sink_facebook_lead_ads_proposal.md based on the sink_proposal_example.md under vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
3. Copy one of the sink templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as sink-facebook-lead-ads.
4. Implement your connector and submit a PR.
5. Wait the connector to be reviewed and approved.

How to claim to implement this connector

f you want to solve this issue, please leave a comment in this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

abc

abc

[Good First Issues]: Review GitHub Source

Documentation Review

Motivation

The best way to learn about an open-source project starts with reading its documentation. In order to better know about Vanus and improve the quality of Vanus documentation, we're looking for OSS enthusiasts to review our docs.

Details

Review and test connectors documentation

Requirements

When you test:

• Use the playground to test GitHub Source
• Run the connector according to the documentation
• Test every link in the documentation

When you review the documentation:

• Grammar check
• Wrong capitalization
• All image are working
• You can understand every step.

If you have any questions or comments, please feel free to leave a comment in this issue.

How to change it:

Comment to Claim this issue.

1. Fork the docs repository.
2. Create a branch Review GitHub Source .
3. Review document GitHub Source
4. Change: Use the playground terminal to test the connector, Review document.
5. Make a PR.

this is a test

test

New Source Connector Request: test Source Connector

this is just a test

Source Connector Twitter

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents and sends it to the target.

• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents, and send them to the target.

• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.

• A push connector is passive and only does its logic when it receives an event
• A pull connector must pull the data in an interval. We use a push method when it’s possible

Features of this Source Connector

Receive events from TikTok when a user’s post is liked, shared, or commented on, and when a user is followed and put the data into a CloudEvent to send it to the target

Specific data needed

When someone likes the user's post

• Post ID: The unique identifier for the post that was liked. The post could be a video.
• User ID: The unique identifier for the user who liked the post.
• User screen name: The screen name (i.e., username) of the user who liked the post.
• Like creation time: The timestamp indicates when the like was created.
• Post URL: The URL of the post that was liked.
• Number of likes: The total number of likes the post has received.

When a post is commented on

• Post ID: The unique identifier for the post that was commented on. The post could be a video.
• User ID: The unique identifier for the user who commented on the post.
• User screen name: The screen name (i.e., username) of the user who commented on the post.
• Number of Comments: The total number of replies the post has received.
• Post URL: The URL of the post that was commented on.
• Comment creation time: The timestamp indicates when the comment was created.

When a user is followed

• User ID: The unique identifier for the user who followed.
• Number of Followers: The total Number of Followers the user.
• Follow creation time: The timestamp indicates when the user was followed.

How to create a new connector

• Fork the vanus-connect repo.
• Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
• Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
• Implement your connector and submit a PR.
• Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

Test

[Good First Issues]: New Source Connector Request: Facebook Page

Source Connector Facebook Page

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents and sends it to the target.
• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents, and send them to the target.
• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.

• A push connector is passive and only does its logic when it receives an event
• A pull connector must pull the data in an interval. We use a push method when it’s possible

Features of the Source Connector

The Source connector receives the following events from the user's Facebook Page

• Follow events: When a user's Facebook page is followed
• Like events: When a user's Facebook page is liked
• Message events: When a user's Facebook page receives a message
• Share events: When a user's Facebook page is shared

Specific data needed

Follow event

• User ID: This is a unique identifier that represents the user's personal account on Facebook.
• Timestamp: Indicates the exact time and date that the user follows the Facebook page.
• Subscribe/Unsubscribe: Indicates whether the user has followed or unfollowed the Facebook page.
• Personal information: including name, gender, email address, country and city, personal hobbies, personal profile, personal title, mobile phone number, personal address.
• Follow Count: The total number of followers

Like event

• User Identification: Unique Device Identifier for Liked Users.
• Timestamp: The time when the like occurred.
• Post ID: Unique Device Identifier for the post that was liked.
• Like Count: The total number of likes
• Personal information: including name, gender, email address, country and city, personal hobbies, personal profile, personal title, mobile phone number, personal address.

Message event

• Post ID: The Unique Device Identifier of the post to which the message belongs.
• Timestamp: The time when the message occurred.
• Message ID: Unique Device Identifier for the message.
• Message content: including text, links, labels and emojis of the message.
• User Identification: Unique Device Identifier for the user leaving a message.
• Reply object: If the message is a reply to another message, you can get the ID and content of the reply object and other information
• Personal information: including name, gender, email address, country and city, personal hobbies, personal profile, personal title, mobile phone number, personal address.

Share events

• Shareer ID: Unique Device Identifier for the user who shared the post.
• Timestamp: Share the time that occurred.
• Post ID: The Unique Device Identifier of the shared post.
• What to share: Text, links, hashtags, emojis, etc. added when sharing.
• Sharing target: If the sharing is carried out through private message or group , etc., you can get the ID and name of the sharing target and other information.
• Share Count: The total number of shares
• Personal information: including name, gender, email address, country and city, personal hobbies, personal profile, personal title, mobile phone number, personal address.

How to create a new connector

• Fork the vanus-connect repo.
• Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
• Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
• Implement your connector and submit a PR.
• Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

test issue

this is a test

🎯 New Sink Connector Request: APITable

Sink Connector - APITable

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Sink Connector

A sink connector receives CloudEvents and does some specific business logic. For example:

• MySQL Sink extracts useful data from CloudEvents and writes them to a MySQL database.
• Display Sink prints incoming CloudEvents out.
• Slack Sink extracts data from CloudEvents and sends them to a Slack channel.

Features of this Connector

• Transform incoming Cloud Events compatible with API Table schema

How to create a new connector

1. Fork the vanus-connect repo.
2. Create a sink_google_sheets_proposal.md based on the sink_proposal_example.md under vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
3. Copy one of the sink templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as sink-google-sheets.
4. Implement your connector and submit a PR.
5. Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

test

test
test
test
body
body
body

[Good First Issues]: New Source Connector Request: TEST

THIS IS THE BODY

[Good First Issues]: update connector doc "connectors/source-mongodb" - 1 point

Motivation

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors.

Reviewing Vanus connector docs will be a good start if you decide to know more about Vanus Connect.

Details

Update the improper image sources in the documentation of source-mongodb.

How to change

1. Fork the vanus-connect repository.
2. Create a branch update-source-mongodb
3. Update file connectors/source-mongodb/README.md
4. Remove the line please see [Event Structure](#Event Structure) to understanding it.
5. Submit your PR

Contribution points

Each issue counts specific points based on its difficulty. Contributors earn the corresponding points if they solve that issue. Contribution points are used to describe the contributions contributors have made. They can also be used to get rewards when there are community events.

This issue counts 1 point.

How to claim to solve this issue

If you want to solve this issue, please leave a comment in this issue like:

I'd like to solve the issue, please assign it to me.

Vanus community will assign the issue to you on time.

[Good First Issues]: update connector doc "connectors/sink-tencentcloud-scf" - 1 point

Motivation

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors.

Reviewing Vanus connector docs will be a good start if you decide to know more about Vanus Connect.

Details

Update the improper image sources in the documentation of sink-tencentcloud-scf.

How to change

1. Fork the vanus-connect repository.
2. Create a branch update-sink-slack
3. Update file connectors/sink-tencentcloud-scf/README.md
4. Change image source https://github.com/linkall-labs/vance/blob/main/connectors/sink-tencentcloud-scf/scf-log.png to https://github.com/linkall-labs/vance/blob/main/connectors/sink-tencentcloud-scf/scf-log.png?raw=true.
5. Submit your PR

Contribution points

Each issue counts specific points based on its difficulty. Contributors earn the corresponding points if they solve that issue. Contribution points are used to describe the contributions contributors have made. They can also be used to get rewards when there are community events.

This issue counts 1 point.

How to claim to solve this issue

If you want to solve this issue, please leave a comment in this issue like:

I'd like to solve the issue, please assign it to me.

Vanus community will assign the issue to you on time.

test

test body

support multiple chat group in Feishu Sink

support multiple chat group in Feishu Sink

🎯 New Source Connector Request: Twitter

Source Connector Twitter

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conform to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Source Connector

A Source connector obtains data from an underlying data producer and delivers it to the target after the original data has been transformed into a CloudEvents. For example:

• MySQL Source: extracts data from the Binlog, transforms it into CloudEvents, and sends it to the target.

• GitHub Source: uses a Webhook server to receive the event from GitHub, transform them to CloudEvents and send them to the target.

• AWS S3 Source: Uses the AWS SDK to get the data, transforms it into CloudEvents, and sends them to the target.

There are two types of connectors push or pull connectors.
A push connector is passive and only does its logic when it receives an event, and a pull connector must pull the data in an interval. We use a push method when it’s possible.

Features of this Source Connector

• Receive events from Twitter when a user’s tweet is liked, retweeted, quote-tweeted, DMs, tweet comments and put the data into a CloudEvent to send it to the target
• Each event type (like, dm, comment, etc ...) Should be specified in the event type specification of the CloudEvent.
• Configuration parameters

Specific data needed

When someone likes the user's tweet.

• Tweet ID: The unique identifier for the tweet that was liked.
• User ID: The unique identifier for the user who liked the tweet.
• User screen name: The screen name (i.e., username) of the user who liked the tweet.
• User name: The full name of the user who liked the tweet.
• Tweet text: The text of the tweet that was liked.
• Like creation time: The timestamp indicating when the like was created.
• Tweet URL: The URL of the tweet that was liked.
• Number of likes: The total number of likes the tweet has received.
• Number of retweets: The total number of retweets the tweet has received.
• Number of replies: The total number of replies the tweet has received.

When someone retweets or quote retweets

• Tweet ID: The unique identifier for the tweet that was retweeted.
• User ID: The unique identifier for the user who retweeted the tweet.
• User screen name: The screen name (i.e., username) of the user who retweeted the tweet.
• User name: The full name of the user who retweeted the tweet.
• Retweet ID: The unique identifier for the retweet itself.
• Retweet text: The text of the retweet, if any.
• Retweet creation time: The timestamp indicating when the retweet was created.
• Tweet URL: The URL of the original tweet.
• Number of likes: The total number of likes the original tweet has received.
• Number of retweets: The total number of retweets the original tweet has received.
• Number of replies: The total number of replies the original tweet has received.

When someone Dms the user

• DM ID: The unique identifier for the DM.
• Sender ID: The unique identifier for the user who sent the DM.
• Sender screen name: The screen name (i.e., username) of the user who sent the DM.
• Sender name: The full name of the user who sent the DM.
• Recipient ID: The unique identifier for the user who received the DM (i.e., you).
• DM text: The text of the DM.
• DM creation time: The timestamp indicating when the DM was created.

When someone replies to the user's tweet

• Tweet ID: The unique identifier for the tweet that was replied to.
• User ID: The unique identifier for the user who replied to the tweet.
• User screen name: The screen name (i.e., username) of the user who replied to the tweet.
• User name: The full name of the user who replied to the tweet.
• Reply ID: The unique identifier for the reply itself.
• Reply text: The text of the reply.
• Reply creation time: The timestamp indicating when the reply was created.
• Tweet URL: The URL of the original tweet.
• Number of likes: The total number of likes the original tweet has received.
• Number of retweets: The total number of retweets the original tweet has received.
• Number of replies: The total number of replies the original tweet has received.

How to create a new connector

1. Fork the vanus-connect repo.
2. Create a source_twitter_proposal.md based on the source_proposal_example.md under the vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
3. Copy one of the source templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as source-twitter.
4. Implement your connector and submit a PR.
5. Wait for the connector to be reviewed and approved.

How to claim to implement this connector

If you want to solve this issue, please leave a comment on this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

[Good First Issues]: Review Amazon Billing Source

Documentation Review

Motivation

The best way to learn about an open-source project starts with reading its documentation. In order to better know about Vanus and improve the quality of Vanus documentation, we're looking for OSS enthusiasts to review our docs.

Details

Review and test connectors documentation

Requirements

When you test:

• Use the playground to test AWS Billing Source
• Run the connector according to the documentation
• Test every link in the documentation

When you review the documentation:

• Grammar check
• Wrong capitalization
• All image are working
• You can understand every step.

If you have any questions or comments, please feel free to leave a comment in this issue.

How to change it:

Comment to Claim this issue.

1. Fork the docs repository.
2. Create a branch Review Amazon Billing Source .
3. Review document Amazon Billing Source
4. Change: Use the playground terminal to test the connector, Review document.
5. Make a PR.

🎯 New Sink Connector Request: Mail-chimp

Sink Connector-Mail-chimp

What is a Vanus Connector

Vanus Connect allows you to skip the complex integration with external services by offering out-of-the-box connectors. Each connector acts as a proxy between outside services and your system, enabling users to have faster integrations with no codes. All data produced or processed by our connectors conforms to the CloudEvents specification, which helps enterprises leverage event-driven architecture to do business in the age of events.

What is a Sink Connector

A sink connector receives CloudEvents and does some specific business logics. For example:

• MySQL Sink extracts useful data from CloudEvents and writes them to a MySQL database.
• Display Sink prints incoming CloudEvents out.
• Slack Sink extracts data from CloudEvents and send them to a Slack channel.

CloudEvent

{
"specversion": "1.0",
"id": "53d1c340-551a-11ed-96c7-8b504d95037c",
"source": "source-connector",
"type": "create-campaign",
"datacontenttype": "application/json",
"time": "2022-10-26T10:38:29.345Z",
"data": {
...
}
}

Each action will be according to the event field type:

"type": "create-campaign",
or
"type": "create-tag",
or
"type": "add-update-subscription",
or
"type": "delete-subscription",
or
"type": "remove-subscriber-tag",
or
"type": "send-campaign",
or
"type": "add-subscriber-tag",

Features of this Connector

• Allow users to Create a new campaign:

You can create a new campaign using the Mailchimp API. This includes specifying the campaign type (regular, automated, plain-text, A/B testing, etc.), the list or segment to which the campaign will be sent, and the content of the campaign.

• Allow users to Create tags:

Tags are a way to categorize and organize your subscribers within your mailing list.

• Allow users to Add or update subscribers:

With Mailchimp's API, you can add or update subscribers to your mailing list. This includes specifying the email address, name, and any additional fields you've created in your list. You can also update existing subscriber data, such as their name, email address, or list membership.

• Delete subscribers:

You can use the Mailchimp API to delete subscribers from your list.

• Remove subscriber from tags:

Tags are a way to categorize and organize your subscribers within your mailing list.

• Send to an existing campaign:

This will send an email to all subscribers of that campaign.

• Add subscriber to tag

add an email address to a tag.]

How to create a new connector

1. Fork the vanus-connect repo.
2. Create a sink_Mail_chimp_proposal.md based on the sink_proposal_example.md under vanus-connect/proposals directory. The proposal will introduce how a developer tends to implement the connector in detail. The proposal PR must be approved and merged before going to the next step.
3. Copy one of the sink templates (Golang template, Java template) to vanus-connect/connectors/ and rename it as sink-Mail-chimps.
4. Implement your connector and submit a PR.
5. Wait the connector to be reviewed and approved.

How to claim to implement this connector

f you want to solve this issue, please leave a comment in this issue like:

I'd like to implement the connector, please assign it to me.

Vanus community will assign the issue to you on time.

test

tt

test

test

mysql source connector error: public key retrieval is not allowed