REST Action
  • Updated on 29 Jul 2020
  • 16 minutes to read
  • Contributors
  • Print
  • Share
  • Dark
    Light

REST Action

  • Print
  • Share
  • Dark
    Light

What are Actions in Rivery?

Actions are the ability to connect to anywhere on the web.

The REST Action is the place to send REST requests in order to get data from Web services (APIs etc.) or post data in Web services.

In order to do so, you’ll have to know the URL of the REST request and any other settings which are required in order to submit a REST request for the desired API.


Let’s create our first REST Action in Rivery.

Create a REST Action

  1. Click on “Create new river” from any screen in Rivery and select the “Action” option.
  2. REST Actionimage30

  3. The first step is the “General info”. Insert a name for the river and proceed to the next step called “Action Steps”.
  4. image.png

  5. An action in Rivery can be multi-action or REST Action.
  6. REST Action-image33

    • Multi-action is an action which is composed of multiple actions, and allows to run a process of actions in Rivery.
    • Rest Action is a generic process which can be configured to send any kind of REST request. Click on Rest Action in order to proceed to the REST Action screen.
  7. The REST Action is composed of two major parts: Request & Results.
    In addition, the REST Action has its own Custom Connection, its own Variables and Time Period inputs.
  8. image.png

    1. Request Configuration is the place to insert all the inputs which are related to the request of the REST Action such as the URL, parameters, headers and more.
    2. Results Configuration is the place to insert all the inputs to handle the results of the REST request.
    3. Custom Connection is the place to define the connection of the action. This connection can be reused over time and crossed rivers. All the details of the selected custom connection will be added to the variables of the action and can be encrypted in the connection
    4. Variables is the place to define the variables of the Action. Those variables can be used in any input in the action.
    5. Time Period is the place to define the time range of the rest action. Select the type of the dates (datetime or date), and select the time format which is required for this request.

Creating a dynamic date range

In order to pull a dynamic date range, so it can be scheduled and pull automatically the latest data:

  1. Set the date period:
    image.png

  2. Add two new parameters in the variables section:

image.png

  1. Create a new river and choose the REST river as a source, then choose the desired date range:
    image.png

Rest Action custom connection

Create or select an existed custom connection.

It is possible to leave the connection empty in case it is not required.

The keys in the connection can be encrypted by using the Is Password checkbox.

REST Action Variables

The variables of the Rest action are available in the button in the right side of the river:
Insert as many variables as required to the variables of the Rest action.

    REST Action-image7

Time period

The time period is the place to configure the start date and end date of the request and is available in the right side button of the time period:

    REST Action-image20

Those two parameters will be automatically added to the variables list of the action and can be used anywhere in the action (usually will be sent as parameters in the request)

The time format input decides how those dates, when used in the action, will be sent in the request - this format should be the expected format of the service which the request is being sent to.

If the required format is not in the list of the time formats, it is possible to insert manually the required time format according to python syntax.

REST Action-image13

Request Configuration

The request section contains all the required settings for sending a proper REST request.

In any of the inputs, it is possible to use the variables that were configured in the variables of the action.

  1. API URL: The address of the service you are about to send the request to. It is supposed to be started with “http://” or “https://”. In case the request has parameters Rivery allows to manage those parameters in the Params inputs. So usually the API URL will be until the part of parameters in the request (until the “?” sign)
  2. REST Action-image21

  3. Select the REST method of the request. For now, rivery supports in GET, POST, DELETE requests.
  4. Params - Click on the Params button in order to add parameters to the request.
  5. REST Action-image3

    1. After clicking on the “params” button the next accordion will get visible

    image.png

    2. Click on the add button to add how many parameters that are required in the request.

    3. Important: The key of each parameter must be equal to the name of the parameter that the service is expected to.

    4. The value of each parameter is the exact value which will be sent in the request. It is possible to use action’s variables as values. If using variables it will be possible to decide which values will be sent in the request from outside the action by passing values to those variables click here to read more about variables.

    4. Authentication type - In some requests it is required to add some authentication method to the request. **Rivery action supports basic Auth** . Select the Basic Auth method if it’s required or leave it as “No Auth” in.

    1. Insert the user name and the password of the user with the access to the service. The user name and the password can be other credentials as well (such as key_id and key_secret).

    2. Basic Auth - Should be used if mentioned that the request should be sent with basic authentication.

    5. Headers - The headers that will be sent in the request. (Leave empty in case it’s not necessary)

    1. Click on the “add” button to add as many as required headers.

    2. Start typing key name and a list of available options will be shown. It is possible to add additional keys which are not on that list.

    3. Insert in the value the exact value to be sent in this header. It is possible to use action’s variables here.

    6. Advanced options - The advanced options contain options which are not mandatory when sending a REST request but can be useful in some cases.

    image29.png

    1. Sleep after request - mentions if and for how long Rivery will wait after sending the request.

    2. Timeout request - mentions for how long to wait for a response. If the given time is passed the action will return an error. If that input is empty the default waiting time is 300 seconds.

    Testing the request

    image71.png

    Once the request is initially configured, it’s the best time to try the “test rest action” button. This will send the request with all the configurations inserted in the in the request section and will return the response that was received for this request.

    If there is some error in the response it will be shown here, so you’ll be able to fix the request according to the error details.

    Results Configuration

    The results section contains all the required settings about how to get the results from the response that was received from the REST request.

    1. Results type - Rest Action can return one of the following types of results. Select the one which is required.

      1. Response The action won’t return anything but will only show the response that was returned for the given request.

        If selected, no options will be shown except the advanced options.

      2. Variables The action will return variables which stores values that were returned in the request’s response. This kind of action can be used inside a multi-action or a logic river.

      3. Data The action will return data which can be loaded into a table in a cloud DB.

        An action with results type ‘data’ can be used inside the REST River.

        Make sure that the data is being returned in a supporting structure as mentioned in “Rest action that returns data ” at the end of this documentation.

    Data location

    image9.png

    Insert the key in the response where the results you are interested in are located.

    If the required data is actually the whole response leave the data location empty.

    If the data is located in a nested key, use ‘.’ to indicate it.

    For example: In response {‘results’:{‘data’: [....], ‘status’: ‘success’}}

    The results we are interested in are located in ‘data’ which is inside ‘results’, so the data location, in that case, would be “results.data”.

    Data position

    image14.png

    In relevance for the case, the response is a JSON array and the required results are located in one of the items in that list. The data position should be the position of that item (starting from 0).

    Variables to return - relevant only in case that the results type is variables

    1. Click on “add” to add variables to return.

    2. Name of the output variable to return- Insert the name of the variable to return

    3. Key in the response - Insert the key in the response which contains the value to be stored in that variable.

      Leave it empty in order to insert all the response into that output variable

      Use ‘.’ in case the value for this variable is located in some nested key in the response.

      Example:

      We would like to send a request to generate a token and return this token as a variable called new_token.

      The response is:

      {"auth_res": {"token": "abcde12345", "expired_at":"2017-07-01 22:00:00"}}
      

      In the data location input, we’ll insert auth_res (because the data we are interested in, is located inside auth_res .

      In the variables to return we’ll add a new variable with key: new_token and value token .

    Results details - Relevant only for 'Data' result type.

    image25.png

    Make sure that the data is being returned in a supporting structure as mentioned in “ Rest action that returns data ” at the end of this documentation.

    Select the results structure -

    JSON - If the data is in JSON structure (meaning a list of rows when each row is an object of key-value pairs)

    CSV - if the data is in a table structure. CSV can be the whole response if the response is just text or can be split to key in the response which stores the heading and another one that stores the values.

    1. JSON Structure:

    REST Action-mceclip1

    If the response is returned as JSON data, but the results data are in a CSV structure (as in the example below the toggle button in the screenshot) it is necessary to indicate where the headings of the data are located (fill the key of the JSON response - for example, headers ).

    Note that this option requires filling both 'Headers Location' and 'Data Location' inputs.

    2. CSV Structure:

    image.png

    If selected CSV it is necessary to indicate where the headings of the data are located. If the headers are at the first row of the data, keep the 'First row in values is headers' toggle enabled. If the headers are at different row, disabled this toggle and fill the headers row number.

    Note: The values of the CSV data were configured in the data location input.

    Pagination Details

    * Relevant on Data result type only

    Sometimes services return the data in pages (also knows as chunks).

    Pagination is the mechanism on how to pull all the results from the service in case it being returned in pages.

    Important:

    the selected pagination will be in actual use only when running the action from a REST River.
    When running the action, it will return only the first page as a sample of the data.

    Pagination Methods

    The REST Action supports the following pagination methods:

    image18.png

    Paginate by offset

    image22.png

    Each page will contain data from the given offset. The amount of rows in each page is being decided according to a parameter sent in the request.

    1. Offset start - What is the start position from where to start pulling the data in REST Action process.

    2. Offset key - The name of the key where to send the offset in the request.

    Paginate by page

    image17.png

    The data is being split to pages, and each request contains the page number to pull. Starting from 1.

    Page key - The name of the key to send the page in the request.

    Next page in response

    Each response contains the details of the next page to pull.

    In that method, we need to indicate where to find this information in the response.

    1. Next page type

      1. Static URL with param from the response

        The response contains some value to be sent in the next request in order to pull the next page. The next request can be sent with the initial URL or with a new URL.

      image28.png

      1. Next page key - the key in the response which contains the value to be sent in the next request.

      2. Send next page in key - the parameter’s name in the next request to send the value of the next page.

      3. Next page in new URL - The URL to send the pagination requests with. Leave empty if using the same URL.

    2. Url in response

      The response contains a full URL ready to be sent to get the next page’s data.

    image.png

    Next page key - the key in the response that contains the URL of the next page.

    Stop according to

    Select how to decide when to stop paginate. When using pagination, we have to indicate when need to stop the paging process.

    image26.png

    1. Page size - the pagination stops when the page size is smaller than the value that was inserted to the input of page size.

      image15.png

    2. Key in response - the pagination stops when the response contains a given value in a given key.

      image27.png

    1. Insert the name of the key in the response which is supposed to contain the value. If the data is located in a nested key, use ‘.’ to indicate it.

    2. Insert the value that should indicate to stop the pagination.

    Sleep after each page
    Hold the process for the given amount of seconds.

    It is recommended to use that input in order not overload the service with too frequent requests when using pagination.

    image12.png

    Advanced options

    Fill the following inputs in order to hold the processor exit it in case of any given returned response in the action.

    1. Insert the number of seconds to hold the process. Insert the name of the key in the response that is supposed to contain some value. Insert the value that should cause the process to hold.

    image5.png

    1. Insert the number of seconds to hold the process. Insert the response code that should cause the process to hold.

    image.png

    1. Insert the key in the response that is supposed to contain the value. Insert the value that should cause the process to exit.

    image.png

    1. Insert the response code that should cause the process to exit.

    image.png

    Keys to monitor

    image11.png

    Insert any key in the response that you’d like to monitor during the Action’s run.

    If the Action found that key in the response it will store its value and will return it in the action results and also in the activities screen (when opening the action details).

    The keys to monitor can be useful when the response might contain messages about errors or the runs.

    In that case, if the action failed, thanks to the “keys to monitor” you’ll have some information about the error when opening the action results in the activities screen.

    Testing the results

    image72.png

    Once the results were configured, it is a good time to test the REST Action.

    This test will run the action exactly the same way as it will be run when executed (either as an action or from a multi-action, logic or Rest River).

    This test should consider all the configuration of the Action results, so you should see updated results comparing to the results of the test you ran previously after configuring the Action request.

    Rest Action That Returns Data

    In case that the Action we are about to create is supposed to return data, that data can be loaded to a target table inside a Rest River (Click here to read more about “Rest Rivers - Loading data from Actions”).

    When using that kind of actions that return data it is important to make sure the data is being returned in the supporting structure (limit #2 in the following section).

    Important limitations about that “data” action:

    • The action will return all the available data from the service only when running from a Rest river. When running the action itself (by clicking on ‘save & run’ button) or when it is executed by a multi-action or logic action, the action will return only a sample of the data.

    • The action must return the data in one of the following structures in the order it will be possible to load the data to a target table when being executed in a Rest River:

    • The data that the action returns is a list of jsons (JSON array).

      For example:

      [{“col_1”: “aaa”, “col_2”: “bbb”}, {“col_1”:”ccc”, “col_2”:”ddd”, “col_3”: “eee”} … {}... {}...., ……{}]

      Each json in the array is actually a record in the data.

      The data that the action returns is a CSV structure. It can be in one of the following csv structures:

      • The data is actually a long string that represents the data to load.

        This kind of data can be in some key in the response or can be the response itself (all the response that was returned is the data)

        For example:

        “Col_1,col_2

        “Aaa”,”bbb”

        “Ccc”, “ddd”

        …..

        …..

    • The data is actually composed of two values that represent the values of the data and the headings of the data. Those two values are located in two keys in the response that will be taken by the action:

      • The values should be a list of lists

      • The headings should be a list.

        For example:

        The headings of the data are located in a key called “headers” and are the following list:

        [“col_1”, “col_2”,...., “col_10”]

        The values of the data are located in a key called “values” and are the following list:

        [“aaa”,”bbb”,...... “zzz”],

        [“ccc”,”ddd”,.......”xxx”],

    Multi Array JSON format

    If the returned data you want to map is within a series of arrays, this can be configured within the 'Data Location' field in Results/Data.

    image.png

    The syntax used will be: 
    key[array location].key[array location]
    
    key - ID within JSON results
    array location - first position within the JSON starts at 0 
    * can be used to designate all objects within the array.
    

    Example JSON:

        {
      "results": [
        {
          "event": [
            {
              "Id": "2832aeaa-2721-44e2-925e-f7f9fd0bfa97",
              "appId": 123123123,
              "appName": "Checkout ",
              "asn": "1234",
              "asnLatitude": 31.123512
            },
            {
              "Id": "cd18fbf7-ae01-4ddb-b1ce-26ca0d73c5bd ",
              "appId": 111111111,
              "appName": "Checkout ",
              "asn": "1441",
              "asnLatitude": 32.381231
            }
          ]
        }
      ]
    }
    

    If you'd like to select the fields in the Events array the following will need to be added to the Data location:
    results[0].event[*]

    Finalize the REST Action

    Once the test of the Action returns the expected results, the Action is ready to be used.

    Run it using the button of the “save & run”.

    The Action will run and will return the results by clicking on “show action results”:

    image1.png

    The results of the action will be according to the results type of action.

    If the Action failed, it will return the response that was received during the action’s run.

    If the results type is “data”, so the results will contain a sample of the data.

    If the results type is “variables”, so the results will contain the variable that was returned by the action.

    If the results type is “Response”, so the results will contain the response of the Action.

    An example of “Data” results:

    image10.png

    The results of the action also contain response details which are the status code that was received in the action and any other parameter that was requested in the input of “keys to monitor” in the Action results configuration.

    The Action response details are also located in the activities screen in the details of the river’s run.

    The new Action you have just created can be used in any of the following rivers in Rivery:

    1. Call this action from a REST River (only if this REST Action has “data” results type).

    2. Call this action from a Multi-Action River.

    3. Call this action from a Logic River.

Was this article helpful?