Part 2: How to measure the progress of Recursive Workflows

Follow me:

If you’re a non-believer in the awesome power of Recursive Workflows, I’m here to change your mind. With a pretty simple setup, this is how we can loop through workflows like a warm knife through butter, and visualize the process for the user live. The illustration below is the kind of sweet stuff we can get working with recursive workflows:

Screen Cast 2021 04 25 at 3.26.20 PM

In my last article about Recursive Workflows, we went over the basic way to set them up to run reliably. And indeed, reliably is the keyword here – unlike Schedule workflow on a list and Make changes to a list of Things, the core strength of a Recursive Workflow is that is does not time out. You can trust that it will finish the job no matter how long it takes. It’ll even run forever if you forget to stop it.

Don’t believe me? I once missed setting the Condition on the last step of a workflow in a client app. It kept running for three months making the same change over and over until I discovered it. That’s how reliable they are (and how great I am).

(Before you get your knickers in a knot, the one exception of course, is if you actually spend all available server capacity – in which case any workflow, including recursive ones, will time out. So keep an eye on your capacity).

Not only will it finish, but because you set the time in-between the cycles yourself, you can pretty reliably predict when it will finish. If you run a simple workflow 60 times with 1 second between each cycle, it will finish in roughly one minute, with the complexity of the actual action steps being the only unknown in the calculation.

All that being said, there are still of course many reasons for why you would want to know exactly when a recursive workflow is finished, or even how far along it is. Actually, recursive workflows allow you to provide more information about the state of an ongoing process than a regular Bubble workflow.

Before diving into this one, I highly recommend that you read the first part of the series, as we’re building on the foundation from it.

Now let’s have a look.

Why track the progress of a recursive workflow?

This may seem obvious, but let’s talk about it anyway. As it often is with Bubble, there are two different perspectives to think about here:

The developer perspective

As the developer, you have concerns that your users may not have. Will the workflow finish at all? Can I learn how fast a workflow typically finishes? Will this one make it in time for the launch tomorrow? When you work on complex apps, you’ll find that you need to run recursive workflows without your client’s or users knowing. Maybe a change in the app needs an update to a long list of records? Maybe you made a mistake somewhere that affects hundreds or thousands of users?

The point from the developer perspective is that you’ll often need a reliable way to tell whether an operation is successful or not, and keeping an eye on its progress is a good way to learn how your decisions affect your app’s performance.

The users perspective

The needs of the users are different. You may want to visualize the progress of a certain task in a progress bar, show how many records have been completed or send an email when the task is done.

It’s usually the case when you use a recursive workflow that you want to process a lot of records, and in many cases you’ll have to make a choice on how to communicate that. I go into detail in my book about the need to control the user’s attention to keep them happy when a task is being processed.

Ways of measuring the status of a recursive workflow

Ok, so knowing who would actually want to know the workflow’s status, let’s look at different ways that we can track the progress to communicate an ongoing status update:

We’re tracking this……to communicate this
When the workflow is finishedTask is not finished / Task is finished
How many records have been processed67 out of 250 records have been completed
The percentage of records finishedThe task is 18% completed
How long the workflow took to completeThe task was completed in 3 minutes
Predicting when the task will be completedThe task will be completed in about two minutes

Setting up the structure in Bubble

For all this to work, we’ll introduce a new Data Type and extend the Option Set from part 1. The Data Type will help us track the progress as the workflow is chewing on its tasks, and the Option Set we’ll use to store some static values.

Setting up the Workflow log data type

The purpose of the Workflow log, as the name suggests, is to save a record of the progress of a given workflow. So create a new Data Type called Workflow log, and set it up with the current fields:

Field nameTypeWhat we’ll use it for
WorkflowType (optional)Option setTo separate different types of workflows from each other
recordsNumberThe total number of records to be processed
recordsProcessedNumberThe number of records that have been processed
startTimeDateThe timestamp when the workflow starts
endTimeDateThe timestamp when the workflow is finished

Setting up the Workflow Delay option set

This Option Set serves the same purpose as the one we set up in part 1. We used it to store three different levels of task complexity: Low, Medium and High. But for this round, we’ll add one more field, called processingTime:

Field nameTypeWhat we’ll use it for
delayNumberThe number of seconds between each cycle of the workflow, i.e. 1 sec
processingTimeNumberThe approximate time it will take to finish the workflow, i.e. 0.3 sec

The value in processingTime is not that important right now, as you’ll have to observe what value makes sense in your app. Your option set should now look like this:

Screenshot showing what the workflowDelay Option Set should look like

Setting up the Back-End Workflow steps

This part is mostly the same as we did in part 1, but with some key difference. In the end, all we’re adding is two more steps, and that’ll give us access to all the information listed earlier. Holy smokes! So let’s get it done.

Adding the parameters

We’re gonna make some slight changes from our last setup to make sure we can track the progress. First, create the back-end workflow and name it:

Animation showing how to create and name the back-end workflow

Set up the parameters as illustrated below:

Screenshot with arrows showing how to set up the parameters for the workflow

Setting up the workflows

The first step of the workflow you’ll recognize from part one. All we’re doing is making a change to the first User on our list:

image 3

Now for our next step, we’re going to save changes to our log. We’re going to add 1 to the field recordsProcessed. On the first cycle of the recursive workflow, this is going to change it from 0 to 1. For every repeating cycle, we’ll know how many cycles the workflow has processed:

Screen Cast 2021 04 25 at 2.02.32 PM

Now, we’re going to add one more step where we make changes to the log, but for this action we’ll set up a condition that makes sure we only run it once – when we’re at the end of the list:

Gif animation showing how to set up the step that updates the end timestamp of the workflowLog

Lastly, let’s set up the recursive re-scheduling. This is the step you’ll recognize from part one that repeats the workflow as long as there are still records left to process:

Screenshot showing the final step that repeats the cycle.

All in all, your workflow should now look like this:

complete workflow 1

Setting up the front-end

Now that we have recursive workflow ready, let’s set up the front-end to see how we can use our new Workflow Log data type to visualize the process as it’s being completed. Let’s return to our opening GIF and see what we want to achieve. You can set up a similar design for yourself if you want to visualize it in the same way:

Screen Cast 2021 04 25 at 3.26.20 PM

Here’s where we’re starting to see just how we can visualize the simple data we saved in the Workflow Log type. Let’s get the button workflow set up.

GIF animation showing how to create the first step in the button's workflow.

The timeStart field is simply for recording when the process started. In the records field, I’m searching for all the Users in my system and counting them. This way I’m saving the total number of records to be processed.

The next steps are straightforward. First, you’ll need to store the Workflow Log we just created somewhere. In my animation I store it in the group with the soft shadow, so that every element inside of it can reference it as its parent. The third and final step is where we schedule the back-end workflow.

Screenshot showing how to set up the workflow on the button.

Now as you can see from this screenshot from my editor, with the information we now have, you can visualize this progress in all sorts of ways. In this example I’m using the math.js plugin to calculate the percentage, and then reference its value in a standard Bubble Progress Bar.

If you want to trigger something when the workflow is finished, you can simply reference the start and end dates in a Do when condition is true workflow.

image 7

And that concludes part 2 of the series on Recursive Workflows. If you liked this, you can also check out my book on Bubble performance or even buy me a coffee!

Support the site and keep it free ❤️

I love tech startups and the Bubble community, and have made it my mission to try and create content that’s valuable, easy to follow and entertaining.

Creating content next to full-time consulting work is time-consuming; if you’d like to support it and keep the site free for everyone, please consider buying me a coffee or becoming a supporting member.

Buy Me A Coffee

Follow me:
Bubble.io books

Learn Bubble the right way

Our professional Bubble books teach you how to plan, structure and build your applications right from the start.

5-star review stars

More Posts

Leave A Reply

Your email address will not be published. Required fields are marked *


Email icon

Useful articles and tips

Join the mailing list to get guides, opinions and articles on Bubble, no-code, automation and the tech industry.

We don't share your email address with anyone, and you can unsubscribe at any time.

You have successfully subscribed