Skip to main content

5 posts tagged with "mentorship"

View All Tags

· 5 min read

It was a pleasant night. I was waiting for LFX to send acceptance/rejection e-mails. And there it was, "Congratulations! You were accepted to CNCF - Tremor" . It was a great and exciting feeling to start this journey! And here I am, at the end of it, writing this blog. It was wonderful, everything that I expected it to be, and even more so in the 3 months! I am writing this blog about my experience in this mentorship.


My name's Prashant (Also known as Pimmy on the internet), a 2nd-year university student pursuing my Bachelor's degree in Information Technology. This blog will talk about my project experience in contributing to Tremor as part of LFX Mentorship Program Spring 2022.

The Problem

We all hate manual tasks, don't we? No seriously if anyone loves doing things on their own, it's totally fine. Of course not everything can be automated. But in this case, it was something more tedious. Here's a flowchart for basic explanation:


This is how it was done, but manually. Each process had to be checked by someone to ensure a smooth sailing. It was quite the work, and so making a release candidate was never easy.

The Approach

The first thing was to divide the tasks into smaller sections and work on this. As my mentors at tremor always used to say, make notes! Keep documenting stuff, really helps. These notes helped me divide the tasks of the current CI process into individual sets of goal, and then I started working on it.

Now I did have to test a lot, 400+ workflows just to get this finally done. So I will explain how the release process works.

Drafting the release

  1. We select which version we want to release, as shown in the code snippet taken from github actions workflow yaml file.
type: choice
description: "Which version you'd like to release?"
- major (_.X.X)
- minor (X._.X)
- patch (X.X._)
- rc (X.X.X-rc)
- release (removes rc)
required: true
  1. Extract the version input (we want major, minor, patch, etc without the brackets), and bumping cargo packages, as shown below. As you can see I extracted the old version before the bump, and put it into $GITHUB_ENV , which is creating env variables with these values. Similarly done for new version after the bump. They are needed for creating the PR.
      - name: Extracting version from input
run: |
VERSION=$(echo "${{}}" | sed 's/ (.*)$//')
- name: Bump new version in TOML files
run: |
OLD_VERSION=$(cargo pkgid | cut -d# -f2 | cut -d: -f2)
cargo set-version --workspace --bump ${{ env.VER }}
NEW_VERSION=$(cargo pkgid | cut -d# -f2 | cut -d: -f2)
  1. Commit, push, and Pull Request is created automatically with Release tag for the release. From there, the maintainers will do all the necessary reviews, and merge once the CI passes.

Publishing Release

  • So, the Draft Release pull request is merged, great! It automatically triggers the release workflow, which by the way only works if the PR has the Release tag, and ignores all other. This is achieved using the conditional statement:
 if: github.event.pull_request.merged && contains( github.event.pull_request.labels.*.name, 'Release')
      - name: Extract release notes
id: extract-release-notes
uses: ffurrer2/extract-release-notes@v1
- name: Create release
uses: actions/create-release@v1
      - name: Trigger publish crates workflow
uses: benc-uk/workflow-dispatch@v1
workflow: Publish crates
token: ${{ secrets.PAT_TOKEN }}

And that's it for the release!

Publishing crates

The Publish crates workflow is now triggered as mentioned in the previous state. There are 4 main crates to be published, and one job to trigger the draft release workflow for tremor-language-server repo (All automated!). Github actions makes it really great to see which job is interconnected.


With all the crates published, including the language-server which follows the exact same process. Tremor has successfully released a new version! Congratulations!

My thoughts

The tremor community has been extremely helpful in guiding me through the entire mentorship. They have this principle of "Never worry, have fun" that will always stay forever with me, and forward in my career. Special mention for Heinz who mentored me throughout the months and helped me. And to the tremor community in general, my thanks to all of them! I didn't know much about github actions or DevOps in general. But now I can confidently say that I can indeed, make processes boring by automating them. I will continue to engage in open source projects, and guide others to the same, cheers!

· 4 min read


Hi folks, I'm Daksh, a senior year CS student at Indian Institute of Technology, Jammu. This blog talks about my project and experience contributing to Tremor as part of LFX Mentorship Program Fall 2021.

Learning about Tremor

I came across rust in early 2020, and I absolutely loved its design, the syntax and how approachable it was to a beginner. I discovered Tremor while looking for open source projects written in rust. Tremor is an event processing system (think kafka) for unstructured data with rich support for structural pattern-matching, filtering and transformation. Over the summers, I did a few minor PR's. Going through the examples and the docs I could set up Tremor and start hacking on!!

My Project

It is very common in event processing to stream data to a persistent storage engine for later processing or archival purposes. My job was to add connectors to stream data to AWS S3. You may find more information in the github issue.

So what is a connector? A connector is the component of an Event Processing System that provides the functionality of communicating with the outside world. This would enable current, and future users of Tremor to now connect and stream events to any endpoint which supports the S3 API.

AWS S3 Connectors

I would explain the sink via an example. To connect to S3, one would require the s3 credentials. Due to lack of support from the sdk only public-secret key credentials are supported (to be extended once the sdk supports other means for credentials). Tremor would read the key names specified in the config from the environment.

define flow s3demo
define connector s3conn from s3 with
"aws_access_token": "AWS_ACCESS_KEY_ID",
"aws_secret_access_key": "AWS_SECRET_ACCESS_KEY",
"aws_region": "AWS_REGION",
"bucket": "tremordemo",
"min_part_size": 5242880,

define connector files3 from file with
"mode": "read",
"path": "sample.json",

define pipeline s3pipe
define script s3Event
let e = event;
let $s3 = {
"key": e.key
let payload = e.payload;

create script s3Event;

select event from in into s3Event;
select event from s3Event into out;


create connector s3conn;
create connector files3;
create pipeline s3pipe;

connect /connector/files3 to /pipeline/s3pipe;
connect /pipeline/s3pipe to /connector/s3conn;


deploy flow s3demo;
{"key": "key1", "payload": {"event1": "hello1", "key2":[1,2,3,4,5]} }
{"key": "key2", "payload": {"event3": {"nested Obj": ["vec1", "vec2", "vec3"]}} }
{"key": "key3", "payload": {"event3": null}}

This configuration reads the file sample.json delimited by lines for events. The s3pipe pipeline destructures the line contents to set the data for the object to upload and its key as meta-data. The s3-sink would then upload the data to AWS S3 with key set to $key inside the bucket tremordemo or anything that is given in the config

Sample Working

The sink also has the min_part_size configuration parameter. S3 support uploading larger objects in multiple parts. One can send multiple events with the same key consecutively, and Tremor would append the content of all those events, and whenever the content size gets larger than the min_part_size, a part is uploaded to s3. Whenever the key changes or Tremor stops, the upload for the previous key is completed.

Ending Thoughts

I had a very productive and fun time with the Tremor Community. The Tremor principle of "never worry about it" has helped me to deal with clueless moments during this mentorship. I would like to express my regards and gratitude to Matthias, Heinz, and Darach for giving me this wonderful opportunity and helping me develop as an open-source contributor and as a joyful person. A special thanks to Matthias for being there to clarify my doubts and fix my mistakes and for being really helpful. I would continue to be a part of the Tremor Community and hope to engage with more newcomers to open-source. I would wish to be part of future CNCF events. You may see me around at the Tremor Discord.

· 8 min read


Hey, I am Rohit Dandamudi from India, about to complete my undergrad in CSE and will be working as a Software Engineer soon. I will be sharing my expereince at Tremor :)

Main motivation for applying

My work involved writing "Property-based tests for tremor-script" and some of the reasons for applying are:

  • It involved a new type of testing I never heard of
  • Be part of a sandbox project where I can learn and grow with the community
  • The concept of learning Erlang + Rust was very interesting to me and frankly out of my comfort zone, as a person used to Python and web development in general.

New concepts I learned specific to my work

  • Erlang and Rust
    • My work mostly revolved around Erlang and a little Rust and I was completely new to this ecosystem, it didn't help to not find much resources or actively accessible community for Erlang.
    • I took this as a challenge and went through various resources to learn Erlang, functional programming in general and I was able to see why this Language was involved to do the task at hand, my mentor is very passionate about Erlang and shared his thought-process, experience which helped me broaden my knowledge and how to approach any concept while learning something completely new.

· 4 min read


Hey folks, I am Nupur Agrawal, a third year student at Indian Institute of Technology Roorkee. This blog describes my experience of contributing to Tremor, CNCF sandbox project in the 2021 spring chapter of LFX Mentorship Program, under the mentorship of Matthias Wahl, Anup Dhamala and Heinz Gies.

Project Abstract

Tremor is an event processing system originally designed for the needs of platform engineering and infrastructure. It is built for the users that have a high message volume to deal with and want to build pipelines to process, route, or limit this event stream.

At the beginning of the program, I was given walkthrough of the project by Matthias and he patiently explained me the components and working of tremor. Tremor is nicely documented and the docs can be very useful for referring many things.

My Project

My project's aim was to enable tremor to receive and send Syslog Protocol Messages, a standard protocol used to send system log or event messages. It was desired to support both the standard IETF format and the old BSD format via UDP and TCP/TLS. More detailed description can be found here.

· 13 min read


Hello folks! I'm Jigyasa, a final-year computer science engineering student at Indira Gandhi Delhi Technical University for Women pursuing my bachelor's in Technology. This blog is about my experience contributing to Tremor as part of the LFX Mentorship program. i

Learning about Tremor

Tremor is an event processing system for unstructured data with rich support for structural pattern matching, filtering, and transformation. It is built for users that have a high message volume to deal with and want to build pipelines to process, route, or limit this event stream. It has a scripting language called tremor-script and a query language as well called tremor-query or trickle.

I had never worked on an event processing system before this internship. In fact, my first major contribution to open-source was through this mentorship program. To get started with it, my mentor Darach Ennis, suggested me some documents that helped me learn more about it:

/docs/overview/ (deprecated)

/docs/course (deprecated)

Apart from that, learning more about the tremor-query, tremor-script, and going through the workshops in the docs can be really helpful.

The codebase of Tremor is in Rust, and since I had no prior experience with Rust, I started learning the language.