Recursive function to look up multiple occurrences of data in a Firebase/JSON structure

Questions about Thunkable
, ,
1

Hi Power Thunkers!

This is an advanced question… I need to create a generic search function running over a Firebase database, which returns from a initial path a list with all the results found below this path.

The search can be of 2 types: search for a value or a bucket.

For example let’s say I search for the value “John Wick”. A possible result in the list would be:

/CUSTOMERS/0017/NAME

/INVOICES/3465/CUSTOMER_NAME

/HELPDESK/TICKETS/542/RESPOND_TO

Or let’s say I want to fetch all buckets that are equal to “2022-05-10”, the result would be something like this:

/LOGS/2022-05-10

/SCHEDULE/2022-05-10

/SALES/2022-05-10

The list must contain all occurrences no matter how deep they may be in the Firebase structure. So we must do a complete reading of the entire structure below the origin point looking for all possible matches, and inserting in the result list.

I thought of several ways to do this, but as far as I know I’ll have to use a recursive search function, to go down in the structure from the first to the last level of each key, using “get object proprieties” of cloud variables, and them use the “for each” or “count” loop blocks to traverse the data of each bucket, to finally store in a list the results that match the defined search criteria.

However, I ran into the issue that the variables “J” or “I” of the loop blocks are unique during the entire app session, so each time the recursive function is called, the variables lose their previous content when back, so we have an erratic behavior of the function.

Another approach I thought of doing this implementation would be to read all structure data coming from Firebase into a big JSON text, and look for all the occurrences inside, but in this case I have no idea how to retrieve the ascending structure of the data found.

Does anyone have any ideas on how to make something like this work?

Any ideas on how to implement this function would be most welcome!

Regards,

1 Like
2

Hello @paulovaz72
This is possible. The recursive function is a technique that is mostly a programmatic approach more than a platform specific.

It is would be very much easier working through an example that just talking about it theoretically.

Do you have an example to use or shall we go back to the GitHub example I shared earlier?

This is very easy to do.

3

I’m not at all an expert in using recursive functions. Honestly, they impress me and at the same time give me a migraine. But I would think you could set this up as follows:

when screen opens
   set app variable remaining list to [list]

function recursive
   for each j in app variable remaining list do
      [code]
   [call recursive]

When starting the screen/app, you would set app variable remaining list to the full list you’re working with. But when you need to resume the recursion, you would change the value of app variable remaining list to only the portion you still want to check.

You could also do this with a stored variable if you wanted to retain the remaining list when the app is closed.

2 Likes
4

The recursive structure would be something like

function recursive with parameter [list]
      check [list] is not empty
         remove first item from the [list]
         do something with that item
         call recursive with the remainder of the [list] as its parameter
     if list is empty do nothing

The function should call itself with a shorter list every round until the list is empty.

1 Like
5

@muneer, Do you know if is possible to use a for each or count loop blocks inside this recursive function without loose the previous values (the “do something” statement above)…

1 Like
6

@muneer, I haven’t tested whether this approach would work yet, as I’m still thinking about the best way to do it, so I thought it best to ask if anyone else has done something similar before doing this.

I’m suspicious that the loop variables won’t produce the desired effect since the variables it uses are global.

And the recursive approach is mandatory because the data structure does not have a fixed number of levels deep, and can vary in different levels in each key.

But not to just stay in theory, what I imagined would be something like:

image
image537×553 56.8 KB

What do you think?

1 Like
7

popping out of in the middle of a nested recursion implies being able to pick up where you left off but you can’t because the internal variable, say j, inside a for j loop is not local to the current function but global. once you use the function again during recursion, you can say goodbye to the previous value of j. there’s no way to go back where you left off.
unless you use a stack to push your latest value of j before you do a recursive call to .then pop it from the stack after you exit. it can be done but it’s a little hairy. and i was only thinking of integer/ordinal values of j. but once you are using object_id’s it’s a different story. plus you won’t be able to use the for block as is - youi’ll have to code your own initialization, end-loop checking,etc.

2 Likes
8

@manyone, this is exactly what I was imagining, because Thunkable doesn’t have variables with limited scope (only visible inside a function for example, like other languages do).

So to solve this, I think the only way is to save another list with the last value of the variable and reset it when you return to the loop… if that works… maybe use the “repeat while” block to avoid use fixed variables and using this approach of stack lists…

I’m a little confused, but the way is to try…

I don’t believe it’s so easy @muneer :slight_smile:

1 Like
9

you can use @muneer’s pseudocode as a template to make it work but the biggest challenge is to code it without using any for loop, or any loop of any kind.

1 Like
10

@manyone , I have no idea how to read the all data from Firebase without use the native Thunkable loop blocks…

Maybe if exists some API to send the Json and recover the keys with the desired results of search, or as I told above, read the entire Json as text and recover the keys…
But at this moment I’m lost…

I’m working in a generic search tool to find data spread over a Firebase database, and I really need to find an approach to get it working.

As I told, it’s an advanced level question… I’m not sure if the masters here already did something like this before.

1 Like
11

See this demo
https://x.thunkable.com/projectPage/627a94304973490012baae64

I used two loops in this project. A normal for each j and a recursive function.

The idea of recursion is to perform the function of a loop without actually looping the code.

12

This particular condition can be solved by renaming the j in the outer loop and the inner loop like this

image
image375×203 8.86 KB

This way the outer loop will return its value every time the inner loop is executed.

13

presumably those blocks are part of a function, say process_list. let’s say inside the inner for, you have to recurse to call process_list. once you do that, the new instance of process_list will re_use outer j and inner j and the old values will be overlaid because they’re not local to the function - they’re global.

1 Like
14

The challenge here is that we don’t know how many levels deep we need to go doing loops…

The idea is to enter in each bucket / subbuckets of the Firebase/JSON, reading all data inside and collecting the paths where we find the searched string…

So lets say you have a structure like this:

/CUSTOMERS
    /1
       /NAME: "Marylin Monroe"
    /2
       /NAME: "John Wick"
    /3
       /NAME: "Paul Walker"
/INVOICES
   /350
       /CUSTOMER_NAME: "Paul Walker"
   /351
       /CUSTOMER_NAME: "Marylin Monroe"
   /352
       /CUSTOMER_NAME: "John Wick"
/SUPPORT_TICKETS
    /1
       /REPLY_TO: "John Wick"
       /ANSWERS
             /2022-05-08
                  /ANSWERED_BY:"Marylin Monroe"
             /2022-05-10
                  /ANSWERED_BY: "John Wick"
    /2
       /REPLY_TO: "John Wick"
    /3
       /REPLY_TO:"Paul Walker

So in this example If I search for “John Wick” I need to have a list with:

/CUSTOMERS/2/NAME
/INVOICES/352/CUSTOMER_NAME
/SUPPORT_TICKETS/1/REPLY_TO
/SUPPORT_TICKETS/1/ANSWERS/2022-05-10/ANSWERED_BY
/SUPPORT_TICKETS/2/REPLY_TO

Please note the Firebase Structure can be changed to include other buckets, adding more data or levels below.

That’s why the recursive function with loops is challenging since I need to read each key, and for each key I can have more keys inside, with more sub-keys until the deepest level of data.

I don’t see how your example applies to this situation…

1 Like
15

See this demo which gets rid of the first loop in my previous demo and makes the output more responsive.
https://x.thunkable.com/projectPage/627abbc3d6c83e00122b9cc9

16

You should not think of loops, instead think of conditions that will require you to go deeper or exit the function,

17

The conditions are:

If the value found is a Bucket, you need to enter on it and search inside.

But if the object isn’t a bucket (it means you reach the deepest level for that object), you just compare with the search string and add the path in the list. After this check, you move to the next object, if exists.

To exit the function, you must read ALL data inside the bucket before leave.

Honestly I don’t know how to do it without using recursive loops…

Either I’m too stupid, or I’m not able to think outside the box after a day working on this. :crazy_face:

18

this works because it follows your template: remove the first item from the input list and process it, then do the function again but for the remainder of the list.
For-loops are not used and no function block ever uses the exact same parameter - it’s always a different version at each iteration and it stops when there’s no more to do. (list is empty).

1 Like
19

i think what you need is to traverse the whole json object by node, each time you do the action you desire (eg. gsearch for some string), each time collecting the values at each successful search, until the end.

in this case, you need the json string (of WHOLE database) to be in memory and it needs to be processed like this (in javascript)

// Implementation of Traverse
function* traverse(o, path=[]) {
    for (var i in o) {
        const itemPath = path.concat(i);
        yield [i,o[i],itemPath,o];
        if (o[i] !== null && typeof(o[i])=="object") {
            //going one step down in the object tree!!
            yield* traverse(o[i], itemPath);
        }
    }
}

// Traverse usage:
//that's all... no magic, no bloated framework
for(var [key, value, path, parent] of traverse({ 
    foo:"bar",
    arr:[1,2,3],
    subo: {
        foo2:"bar2"
    }
})) {
  // do something here with each key and value
  console.log(key, value, path, parent);
}

your json object goes in the area where foo"“bar” is found,while the action to be done is where the console.log(…) is found above.
it would be nice if an API could do this functionality so that at each call, you get a fully “path”-ed object with these elements (key, value, path, parent) and you simply apply your soure selection action on it.
(found this here: Traverse all the Nodes of a JSON Object Tree with JavaScript - Stack Overflow)

4 Likes
20

This is a typical recursive function which, in the old days, we used to do a similar one to list the files in a particular directory in the computer along with its subdirectories.

The solution works because you can easily determine if the item being processed is a file or a directory.

In Thunkable there is not block to check component or variable type.

If you can check if this is an object or a value-key pair then it can be solved.

[Update]

This is my unfinished work. The first step is to traverse to the end of the list.
Thunkable (please ignore this, I will delete the project)

[Update 2]
See my solution. It can still be optimized but my target is a working version of the recursive search.
https://x.thunkable.com/projectPage/627b9413c773c400128cb193

@paulovaz72
try it and let me know.
You can add to the JSON block in the project and you will see that the function will traverse to new entries.

2 Likes