Introducing the New Workflow Execution Engine

As the CEO of Trickest, I am delighted to reveal a project that our dedicated team has been passionately working on for more than a year in response to valuable feedback from our users. This journey has been filled with challenges, innovation, and anticipation, and we're excited to finally share it with you.

Introducing the Trickest's New Workflow Engine—launched on April 24th at 12 PM UTC. This state-of-the-art execution engine substantially elevates the speed, scalability, and efficiency of your workflows, resulting in a safer and more secure digital landscape for all. Embrace the power of the New Workflow Engine and read further about the transformative impact on your pentesting and bug bounty hunting endeavors.

Why we built it?

Trickest platform and its constantly growing user base need a faster, more powerful, and scalable solution for workflow execution. Our goal is to develop an innovative execution engine that offers remarkable improvements in performance, scalability, and flexibility, ultimately empowering offensive cybersecurity professionals to combat complex threats with greater effectiveness.

Previous workflow engine was built on top of open-source libraries and products such as Rancher, Kubernetes and Argo Workflows, which over the time turned out to be slow, unstable, unscalable and sometimes very costly for our users.

While Kubernetes and Argo Workflows are undoubtedly valuable tools in the industry, we found that their capabilities are not sufficient to meet our unique automation requirements and maintain the desired pace of development.

We strived to develop a workflow capable of searching the internet for web servers in just one day.
We aimed to create a single workflow with the capacity to capture every SSL certificate ever issued.
We sought the ability to scale up to 300 machines in just a minute.
We wanted to brute-force millions of hosts within a matter of minutes.
We dreamed about a system of simultaneously scanning millions of GitHub repositories for secrets.

We envisioned a future where conducting internet-wide research is effortless, affordable, and accessible to everyone!

New Engine Capability

Self-Hosted Version of Trickest

By moving away from open-source libraries and making our proprietary execution engine, we are on a fast pace for self-hosted deployments offering the same unlimited scalability for companies of all sizes.

Machine Resources Management

By replacing Kubernetes, resource usage on the machine is significantly reduced, enabling execution machines to be x2 smaller for the same tasks, and reducing the machine's price significantly for workflow executions.

Lightweight agent and direct machine scaling enable workflow startup time to be reduced from 5 minutes to 10 seconds regardless of how many machines users want to execute it on. This improvement holds true if you wish to use 5, 10, or 50 machines, providing unparalleled flexibility.

System Stability & Flexibility

By abandoning the cluster-way of managing the nodes, the new engine completely isolates the execution of machines and single points of failure where the clusters malfunctions are gone. Without services overhead, the tools and scripts within the workflows use 95% of the machine resources, making it almost the same as the local executions.

Tools & Scripts Startup Time

Every time the workflow was executed, the tools and scripts were being downloaded (docker pull-ed) over and over again resulting in significant costs and resource usage. The new engine has the tools pre-installed, making the execution start in a blink of an eye.

Inter-Node Communication

With the large-scale outputs, plenty of time of machine resources was wasted by constantly uploading and downloading the data between the nodes in a workflow. The new engine capabilities support hard-drive mounts making the data transfer instant.

Future major projects

Workflow Debugging & Pausing

New features on the engine consist of debugging & pausing of the workflows, which enables seamless workflow development and makes cybersecurity engineers even more productive.

Reserved IPs

Company users will soon be introduced to a feature to reserve static IP addresses and have better control of their adversary simulation and attacks.

Table Comparison

Feature	v1	v2
Setup Complexity	Complex with Rancher/Kubernetes & Argo Workflows	Streamlined proprietary engine
Resource Usage	Resource-intensive default services	2x decrease in memory and CPU usage
Startup Time	~5 minutes	10 seconds
Tool Availability	Downloaded on execution	Pre-installed tools
Resource Throttling	Limited due to Kubernetes services	Utilizes 95% of machine resources
System Stability	Kubernetes instability	No Kubernetes clusters
Machine Flexibility & Distribution	Limited configurations & geographic distribution	Easy deployment of different types & locations
Inter-Node Communication	Slow communication between nodes	Instant transfers regardless of file sizes
Parallel Execution	Limited to 500 parallel executions	Hyperscalable parallel execution
Fault Tolerance & Job Management	Lack of fault tolerance in Kubernetes	Distributed, fault-tolerant logic
Self-Hosted & Single Machine Support	Incompatible	Supported without third-party software
Pricing	Hour-based pricing	Minute-based pricing
Debugging & Workflow Control	Difficult with open-source libraries	Soon: In platform workflow debugging and pausing features
IP Address Management	Challenges in fixed IP assignments	Soon: reserved elastic IPs

|

Conclusion

The new workflow execution engine brings a paradigm shift in cybersecurity by providing a comprehensive solution that addresses the pain points of the old engine. With significant improvements in performance, scalability, and flexibility, cybersecurity professionals can now respond more effectively to threats and vulnerabilities in an ever-evolving digital landscape.

By simplifying deployment and management, maximizing available resources, and offering enhanced debugging and workflow control, the new engine empowers cybersecurity experts to focus on their core tasks. With features like rapid startup times, instant access to tools, improved system stability, and support for various deployment options, the new engine is well-equipped to meet the growing demands of the cybersecurity industry.

The upcoming blog posts will showcase a detailed workflow execution comparison between the old and new engines, highlighting the key differences and improvements. We will also cover various use cases that were not feasible before the new engine, illustrating its versatility and potential to revolutionize the cybersecurity industry.

Sign-in to immediately test the new engine or Register and discover a tool that will make your life easier!