Data-flo
Data-floSource CodeCGPS
  • INTRODUCTION
    • What is Data-flo
    • Getting Started - Sign In
    • Privacy and Terms Of Service
    • Contact - Help & reporting errors
    • Change log
  • USING DATA-FLO
    • Data-flo site navigation
      • Transformations Page
      • Run Page
      • Canvas
    • Data
      • Bringing data in to Data-flo
      • Getting data out of Data-flo
      • Data Types
        • Boolean
        • Datatable
        • File
        • Graph
        • List
        • Map
        • Number
        • Text
    • Regular Expressions (RegEx)
    • Adaptors overview
      • Components of an adaptor
      • Binding types
        • Bind to Data-flo input
        • Bind to value
        • Bind to another transformation
    • Specific adaptors
      • add-column
      • append-to-list
      • calculate-time-difference
      • change-column-case
      • columns-concatenation
      • concatenate-text
      • create-microreact-project
      • csv-file-to-datatable
      • csv-to-datatable
      • datatable-columns
      • datatable-to-csv-file
      • datatable-to-graph
      • datatable-to-list
      • datatable-to-map
      • datatable-to-sqlite-file
      • date-to-text
      • dbf-file
      • dot-to-graph
      • download-file
      • dropbox-file
      • epicollect-project
      • extend-datatable
      • figshare-file
      • file-to-text
      • filter-blank-values
      • filter-columns
      • filter-list
      • filter-rows
      • filter-rows-numerically
      • force-directed-layout
      • format-date-column
      • forward-geocoding
      • ftp-file
      • gather-rows
      • google-drive-file
      • google-spreadsheet
      • graph-to-dot
      • join-datatables
      • list-to-datatable
      • lookup-map-value
      • merge-datatables
      • merge-lists
      • microreact-project
      • mysql-database
      • newick-leaf-labels
      • oracle-database
      • postgress-database
      • prepend-to-list
      • remove-columns
      • remove-duplicate-rows
      • rename-columns
      • replace-blank-values
      • replace-column-values
      • replace-text
      • replace-text-in-list
      • replace-value
      • reverse-geocoding
      • row-column-value
      • s3-file
      • select-columns
      • send-email-message
      • slice-datatable
      • slice-list
      • smb-file
      • sort-datatable
      • sort-list
      • split-column
      • split-datatable-rows
      • split-list
      • split-text
      • spread-rows
      • spreadsheet-file
      • sql-server-database
      • sqlite-database
      • sum-rows
      • text-template
      • text-to-file
      • unique-list-items
      • update-epicollect-entries
      • update-microreact-project
      • update-smb-file
      • upload-file-to-google-drive
      • upload-files-to-google-drive
      • url-builder
      • yaml-to-json
    • Building a data-flo
      • Debugging mode
      • Show detailed errors on Run Page
      • Permissions - Access Control
    • Tips & Tricks
  • TUTORIALS
    • Prep outbreak data for Microreact
    • Common use cases, solved
      • Fixing datatable headers
      • Select, remove, rename, reorder columns
      • Data in separate files
      • There's no single-column unique row ID (primary key)
      • Ensure non-dates stay non-dates
      • Connect directly to a database
      • Access files on a drive
Powered by GitBook
On this page
  • Recommended Build Process (using Debugger)
  • Tips
  • Example
  • Missing or erroneous input arguments
  • Run Page warning: This dataflow is incomplete
  • Missing adaptor input
  • No data-flo outputs
  • Garbage Collection

Was this helpful?

  1. USING DATA-FLO
  2. Building a data-flo

Debugging mode

Use the debugger to test a data-flo as you build it

PreviousBuilding a data-floNextShow detailed errors on Run Page

Last updated 2 years ago

Was this helpful?

Debugging Mode is a way to run the data-flo from the Canvas instead of from the Run Page.

Using the Debugger is especially helpful as you build a new data-flo, or when updating a complex data-flo. You can look at the intermediate outputs as each sequential adaptor is added, and as the steps occur, to ensure that you are getting the results you want and expect.

Unlike Run Mode, Debugging mode tracks all outputs and intermediate pieces.

Recommended Build Process (using Debugger)

Add your first data import adaptor. Run the data-flo in Debugging mode. If it succeeds (adaptor card turns green, not red)

Test periodically as you build. Whenever you run the Debugger and get output(s) that are a viewable data type (e.g. data, list, text, map, etc), click on the output to see it in the Debugger panel. This allows you to check whether it looks the way you expect.

Tips

TIP: Build left to right, and add adaptors to the canvas in the order they should run. This will reduce unexpected errors.

TIP: Don't close the Debugger between test runs, or you'll have to enter the inputs again. Keeping it open leaves file paths and other input values alive, making it faster to rerun the Debugger.

TIP: If you keep getting errors, click the Save (disk) icon next to the Debugger icon, and refresh your browser page. Run the Debugger again.

Example

Missing or erroneous input arguments

Two errors: If no input is specified for a required argument, or if an impossible binding is set.

Run Page warning: This dataflow is incomplete

Missing adaptor input

When the data-flo contains at least one adaptor without a specified input, the Run Page will show the error, and the Debugger will also throw an error.

No data-flo outputs

When the data-flo has no adaptor outputs marked as data-flo outputs, the Run Page will show the warning, but the Debugger will not show an error. This is because the Debugger is meant to be used on incomplete data-flos to assist in testing intermediate steps.

Garbage Collection

When you rerun the data-flo in Debugging Mode, the software starts fresh with what it's tracking, but your browser and some of the Data-flo Javascript has various caches and related garbage collection that you (and Data-flo) can't control or force-empty. Occasionally, this junk causes problems, leading to errors even when the data-flo should work. The rule of thumb is that refreshing the whole page should help, and browsing elsewhere before returning should also solve the problem. Thus, "turn it off and on again" makes sense here.

Occasionally, you may need to refresh the whole page to clear the browser's cache, to stop unexpected errors from persisting.

TIP: "" (an option in the right sidebar for Data-flo details) does not affect the Debugger messages. It is used to show more detailed errors on the Run Page.

Show detailed errors
garbage collection
Using the Debugger
Debugger toggle, Debugger Panel, "Show detailed errors" option
Dataflow Debugger error messages
Run Page warning: This dataflow is incomplete