Importing Trips & Ticks into J-Bird (Version 0.5.2)

Summary

Overview

You can import sightings into J-Bird from a few types of sources. All sources are tables in which rows represent observations to be imported and columns contain information about the observations. The types of columns in the table depend on the types of observations you wish to import. To import observations for a single trip, the table might include a column of species names, a column of observer names, a column of counts, and a column of notes. The column of observer names is optional if there is only one observer. The column of counts is optional if birds were not counted. The column of notes is optional if there is no note. To import observations from more than one trip, the table should contain columns that identify trips with which observations are associated. Trips in J-Bird are identified by date, region and locality. A table that contains observations from trips in more than one locality in more than one region on more than one date should contain all three columns. Columns can be omitted if all values in them are the same. For example, the column of region names can be omitted if all trips in the table are from the same region. Tables can be extracted from files of comma-separated values (CSV), Excel spreadsheets, tables in web pages, dBase .dbf files, and relational databases.

When you import observations, you'll identify the source of data and describe the layout of the table. That is, you'll be asked to provide the file, web page or database that contains the table and asked to identify columns in the table. As the data are imported you'll be asked to identify species in the data that are absent from the master list of species by matching the species in the data with one in the master species list. It is not presently possible to add species to the master species list.

J-Bird provides two opportunities to import observations into selected trips and one opportunity to import observations into many trips. You can import observations into a trip when the trip is first created. A button is provided in the dialog that you use to enter information about the trip. Alternatively, you can import observations into a trip by selecting it on the tree of trips and then selecting "import" from a pop-up menu that appears in response to a right-click (ctrl-click on Mac OS X). For importing ticks from many trips, a button is available from the task "create/edit trips and ticks".

The process of importing observations has been organized into several steps. First, you will identify the source of data (e.g., file, web page) to J-Bird. J-Bird will read the data and show them to you. If the data are in an Excel spreadsheet, you'll be asked to select a spread sheet from the file, or if they are in a web page table, you'll be asked to select a table. Next, J-Bird will show you the data in table form, and you will tell it which columns in the table contain which information. You'll also have an opportunity to edit the table before it is imported. Then, J-Bird will begin importing. If it encounters species that are not in the master list of species, it will present you with some possibilities. If you are importing scientific names, J-Bird will list all species in the genus to cover the possibility that the specific epithet has changed. Next it presents species with similar specific epithets on the chance that the species has been moved to a different genus and the specific epithet remains unchanged or has a new suffix. If you are importing common names, it will present you with a list of common names that begin with the same letters as the species to be imported. Finally, when the import is complete, you'll see a summary of the action that includes box scores and a list of observations that could not be imported.

J-Bird takes the following steps to identify imported species. It stops at the first successful step.

  1. Query master list for exact match of scientific name.
  2. Query master list for exact match of common name.
  3. Query species synonyms for scientific name and common name.
  4. Query genus synonyms for exact match of genus and query master list for exact match of specific epithet.
  5. Query master list for all species in the genus (or genus synonym), and present the list to the user for selection.
  6. Query master list for all specific epithets that are similar to that given, and present the list to the user for selection.
  7. Query master list for all common names that are similar to that given, and present the list to the user for selection.

Initially, J-Bird contains no synonym. You can build a list of synonyms to minimize the task of identifying species that are not on the master species list. When you identify a species, you can save the name as a synonym of the species that you select from the master list. You also have the option of saving the genus name as a synonym of a genus in the master species list, if you are importing scientific names. If you save a species name as a synonym, the importer should identify that name in all future imports. If you save a genus name as a synonym, the importer should identify any species in the genus for which the specific epithet is identical to an entry in the master list. For example, some lists use the genus Chen for some geese. If species in that genus appear in the master list as members of the genus Anser, they will be correctly identified if Chen exists as a synonym of Anser and if the specific epithets match. Note that genus synonyms will not work for individual species that have been moved from one genus to another because J-Bird forbids genera that are synonyms from also being in the master list.

Data sources

Column types & formats

Species names

Bird names appear in many different forms. J-Bird accommodates some common forms.

J-Bird attempts to convert cases of imported species names to match names in the database.

Observers

A table of observations in which there is one column of observers.
Species Date Observer
Grosbeak, Black-headed 2004-11-1 Gladys
Grosbeak, Black-headed 2004-11-1 Mort
Bunting, Lazuli 2004-11-1
Observers can appear in one of three ways: they can be absent if all imported observations were made by a single observer, present in a single column of observer names, or present in separate columns, one for each observer. If a single column is used, it contains the names of observers, and each row in the table represents one observation by one observer on one trip. If the name of the observer is left blank, the name from the previous row is carried forward (e.g., in the table to the right Mort saw a Lazuli Bunting).


A table of observations in which there is one column for each observer.
Species Date Gladys Mort
Grosbeak, Black-headed 2004-11-1 X X
Bunting, Lazuli 2004-11-1 X
If each observer appears in a separate column, values in the observer columns indicate which observers saw species, and each row in the table represents one species that was observed on one trip. Non-blank entries indicate sightings. Blank entries are ignored. For example in the table to the right, both observers saw the Black-headed Grosbeak, but only Mort saw the Lazuli Bunting.


Counts

A table of observations in which there is one column of observers and a column of counts.
Species Date Observer Count
Grosbeak, Black-headed 2004-11-1 Gladys 14
Grosbeak, Black-headed 2004-11-1 Mort 7
Bunting, Lazuli 2004-11-1 X
How counts appear in tables depends on how observers appear. If there is a single column of observers, counts are entered in separate column. Zeros and blank count entries are interpreted as species that were not seen. Integers greater than zero are interpreted as counts. Anything else is interpreted as a species that was observed but for which a count was not recorded.


Example table that illustrates how to represent counts if each observer has his or her own column in the table.
Species Date Gladys Mort
Grosbeak, Black-headed 2004-11-1 14 7
Bunting, Lazuli 2004-11-1 X
If observers have their own columns, counts may be entered as values in the observer columns as shown to the right. Use blanks or zeros for species that were not observed, integers greater than zero for counts and anything else for species that were observed but not counted. For example, the table to the right indicates that only Mort saw at least one Lazuli Bunting and that he did not record how many he saw.


Dates

Dates in CSV files, HTML tables, and in text fields in either Excel files or relational databases should be in the form YYYY-MM-DD (e.g., 2005-10-20). Excel dates are acceptable in Excel spreadsheets. If a date is blank, the value from the previous row is carried forward. In relational databases fields can be java.sql.Date values, and all rows are expected to contain legitimate date values.

Localities

Enter names of localities just as you would if you were using the user interface to create trips. When a blank value is encountered, the value from the previous row is carried forward.

Regions

Enter names of regions just as you would if you were using the user interface to create trips. When a blank value is encountered, the value from the previous row is carried forward.

Trip keywords

Enter trip keywords just as you would if you were using the user interface to create trips. Keywords are not carried forward.

Notes

Trip notes and species notes are formatted in the same manner.

It is easiest to import simple notes that are single sentences or single paragraphs. The only constraints that apply to notes imported from files of comma-separated values are: newline characters are not allowed, if notes contain commas they must be enclosed in double quotes, and double quotes in the text should be escaped.

Notes that contain more than one paragraph can be imported. The trick is to delineate paragraphs, and this can be done by several methods. The method that is best depends on the data source. A method that works with all data sources is the vertical bar or pipe (|). Just insert it between paragraphs. The most natural method of delineating paragraphs is to skip a line between them (i.e., two successive newline characters), but it is impractical for most data sources. Indeed, it can be used only when importing notes from a relational database. Blank lines can be simulated using the string \n\n. Just insert \n\n between paragraphs. Finally, you can use the <p> marker that is used to mark new paragraphs in web pages. This marker works for data sources other than tables that occur in web pages.

Finally it is possible to import HTML documents as notes from data sources other than tables in web pages. It is probably impractical to import HTML documents from files of comma-separated values because each note must reside on a single line of the file. HTML documents are edited before being imported, and the editing may disturb the nature of the document. Anchors, images and references to style sheets are removed to be sure that the document is self-contained. Tags for frame, frameset, forms, and buttons are removed. The document header is replaced with a very simple header. All paragraph breaks are extracted and replaced. Horizontal rules and explicit breaks are removed.

Starting an import

Ticks can be imported into existing trips. You can import ticks at the time that a trip is created or afterward. When you create a trip, you may elect to import species into it by selecting the "Import" radio button on the dialog into which you enter information about the trip. After the dialog is dismissed, a wizard will appear and guide you through the process of importing ticks.

Ticks can be added to any existing trip on the tree of trips that appears when you select the task "create/edit trips and ticks". Select the trip into which you want to import observations, right-click on it (control-click on Mac OS X), and select "import ticks - ...". A wizard will appear.

Observations for many trips and ticks can be imported by clicking on the button "Import trips and ticks" that appears near the top of the window after you have selected the task "create/edit trips and ticks". A wizard should appear shortly after you press the button.

Identifying the source of data

Dialog for identifying a data source Use the first panel of the wizard to identify the source of data. For CSV files and for Excel spreadsheets, the source will probably be a file on your computer. You can enter the name directly into the text box or by pushing the "Browse files ..." button. If you decide to type the name into the box, the file is assumed to be in your home directory.

Web pages that contain tables from which observations will be extracted may be files on your computer, or they may be pages on the world-wide web if your computer is connected to the internet. You can type the URL of the web page into the text box, or you can browse the web by pushing the "Browse WWW ..." button. Go to the page that contains the table and dismiss the browser. Not all web pages on the internet will work because of the simplicity of Java's web page rendering engine. Links in web pages that run Javascript code will not work. (Java and Javascript are very different despite the similarity of their names.) To avoid this issue, use your favorite web browser to visit the page that contains the table; save a local copy of the page; and use the "Browse files ..." button to locate the saved page.


Identifying table layout

Dialog for identifying a data source Observers may be represented in tables in several ways, and you'll need to let the importer know the layout of your table. You can do so by clicking on the appropriate icon. The simplest table layout has no column for observers. It is useful if all observations are for a single observer. The second layout contains a single column of observer names. Each row in this layout represents one species that was observed by one person on a a trip. The third layout contains a column for each observer.

In the example to the right, the top option has been selected - there is no observer column.

What happens next depends on the data source.


Identifying the Excel spreadsheet

Dialog for selecting a sheet within an Excel file If you are importing from an Excel spreadsheet, the next panel to appear will show the first sheet of the file and tabs for switching between sheets. Use the tabs to select the correct sheet, and click forward.

In the example shown, three sheets are available, and the user has selected the sheet that is named Wednesday.

The next step is Identifying column types and modifying data.


Identifying the table within a web page

Dialog for selecting HTML table To select a table, click on cells in opposite corners of the table. For example, you might click on the upper right cell and then the lower left cell. When you click on the first cell, it will turn green, and when you click on the cell in the opposite corner, it, too, will turn green. All cells between the two selected cells will turn green.

You can select a subset of a table rather than the whole table. Click on opposite corners of the section of the table that you want.

In the example shown, the user omitted the row of column headers by clicking on the date 2004-10-04 at the upper left and the number 1 in the lower right cell. The selected table entries turned green.


Identifying column types and modifying data

Dialog for identifying types of columns You must identify the contents of columns. Columns are identified by the column headings. Initially, all columns are labeled "ignored". To identify columns, click on the column headers, and select from the pop-up menus that appear.

You can choose how to handle entries in the table that are not found in J-Bird's master list. Attempts can be made to match entries in the table with those in the master list based on similarities in species names. Similar names are presented for approval. Alternatively, you can elect to skip entries in the table that are absent from the master list. A list of unknown species will be presented after the table is processed.

In the example shown, the user has identified the first column as a column of dates, the second column as localities, the third column as common names, the forth column for Mort and the fifth column for Gladys. The user has elected to skip any entry in the table that does not match a species in the master list.

Data in the table can be edited. You can change the value of any cell by clicking in it. A cursor will appear. Make the change, and press return or click in another cell. Rows and columns can be deleted. Right-click in a cell (ctrl-Click on Mac OS X) and select from the pop-up menu that appears. Note that there is no undo feature. If you blow it, you blow it, although you can press the Back button, reconfigure and try again. Changes that you make to the table are not transmitted back to the data source.


Watching the import progress

As J-Bird imports ticks, it displays its progress. A status column is added to the displayed table of observations, and it is updated as the import progresses. If J-Bird encounters species names that it cannot resolve, it will present suggestions if it can discern reasonable possibilities otherwise, unresolved names are marked as "unknown" in the status column.

Resolving name mismatches

Congeners of unknown species presented for selection When names of species that are being imported do not match names in the master species list or synonyms, the wizard displays a list of alternatives for your selection if it can. If both scientific names and common names are being imported it tries congeners, species with similar specific epithets and species with similar common names in that order. If the species is identifiable in the set of choices, select the choice and accept it. You can save the name as a synonym by ticking the appropriate check box before accepting your selection. If no species is acceptable, indicate that. Species that cannot be identified will be labeled "unknown" in the status column of the display, and they will be available to view in the summary table as well as in the J-Bird log.

If you are overwhelmed by the prospect of dealing with many mismatches, you can skip all remaining mismatches by clicking on the button "Skip all unknown species".

You can cancel the import by dismissing the window that requests identities of unknown species.

In the example shown, congeners of the unknown American Three-toed Woodpecker (Picoides dorsalis) are presented for selection. The user has selected the Three-toed Woodpecker (P. tridactylus) from the master species list and chosen to save the American Three-toed Woodpecker as a synonym of the Three-toed Woodpecker. It would not make sense to try to save the genus Picoides as a synonym of itself, and if the box were checked, the wizard would ignore it.


How to cancel an import

You can cancel an import while it is in progress by dismissing the import dialog, by pressing the "Back" button or by dismissing the dialog that is used to identify species that are not in the master species list.

Some details

Tips

Example 1: The simplest import

A simple list of species that were observed on a trip.
Pheasant, Common
Bobwhite, Northern
Pigeon, Rock
Dove, Mourning
Hawk, Red-tailed
Flicker, Northern
Starling, Common
Sparrow, House
Cardinal, Northern
Sparrow, Song
Blackbird, Red-winged

Suppose that you have a list of species that you observed on a single trip and that you would like to create a trip and import the list of species. In this example, the table to be imported consists of one column that contains species names. And, we'll assume that the list is in an Excel spreadsheet.

  1. Create a new trip by pressing the button "Create new trip".
    1. In the dialog that appears enter information about the trip.
    2. Just below the notes section, select the radio button that is labeled "import ticks".
    3. Dismiss the window, and an import dialog should appear.
  2. Provide the file that contains the Excel spreadsheet.
    1. Click the "Browse files ..." button and use the browser to select the file.
    2. Click the "Forward button. Next you will be given a choice of table format.
  3. Select the appropriate table format.
    1. In this example, there is just one observer, and the observer's name is not provided in the data file. Select the top entry that indicates that there is no observer column and that all observations will be for a single observer.
    2. Click the "Forward" button. The file will be loaded, and you should see tabs for the sheets in the file along with the contents of the first sheet.
  4. Select the worksheet that contains your data.
    1. Use the tabs to select the appropriate sheet.
    2. Press forward, and the sheet that you selected should appear alone.
  5. Specify the column as a column of common names.
    1. Click on the column header, and select "Common names" in the menu that appears.
    2. Click forward. Next, you'll be asked to identify the observer.
  6. Provide the name of the observer, if the name is not correct.
    1. To change the name click in the cell that contains it.
    2. Type the new name.
    3. Click "Forward", and the import should begin.
  7. Be ready to identify species for which names in the checklist do not match those in your master species list.
  8. When the import has finished, click the forward button to see a summary and then dismiss the wizard.

Example 2: Import a trip with species notes and some carried values

Observations of species on one trip, made by two observers.
PicoidesvillosusGladys
PicoidesvillosusMortAll hairies that we saw today
were working on cavities.
pubescensGladys
Mort
Parus carolinensisGladys
Mort
bicolor GladysOne of the titmice seen seemed
to be prospecting for cavities.
It would fly up to, perch and
look into holes and depressions
that were deep enough to be dark
inside.
Sitta carolinensisGladys
Mort
ReguluscalendulaGladys

Suppose that you have observations from two people on a single trip, observations that you would like to import into a trip that exists in J-Bird. In this example, scientific names are used. Genus names and specific epithets are in separate columns. One column contains names of observers. Notes have been recorded for two species. Each note is a single paragraph. At some places in the table, genus and specific epithet are provided only when they change. We'll assume that data are imported from a file of comma-separated values. CSV format is not shown in the table to the right because it would be too difficult to inspect visually.

  1. Select the trip and import ticks into it.
    1. Navigate the tree of trips and select a trip by clicking on it.
    2. Obtain a pop-up menu for the trip by right-clicking on it (ctrl-click on Mac OS X).
    3. On the menu select "import ticks", and an import dialog should appear.
  2. Select the file that contains the data.
    1. Press the "Browse files ..." button, and use the browser to locate the file.
    2. Press the "Forward" button, and a selection of table types should appear.
  3. Select the appropriate table type.
    1. In this example, there is one column of observers, and each row represents one species that was seen by one observer.
    2. Select the middle table type that is one column of observers.
    3. Press "Forward", and the table of data will appear.
  4. Identify columns types.
    1. For each column click on the header and select the column type on the menu that pops up.
    2. Click "Forward" to begin the import.
  5. Be ready to identify species for which names in the table do not match those in the master species list.
  6. When the import has finished, click the forward button to see a summary and then dismiss the wizard.

Example 3: Import several trips

Observations from two trips. Each observer is represented in a separate column.
DateLocalitySpeciesMortGladys
2004-10-4EllerbeHairy Woodpecker23
Downy Woodpecker55
Brown-headed Nuthatch11
Eastern Bluebird128
Chipping Sparrow44
2004-12-8PinetopsDowny Woodpecker22
Brown-headed Nuthatch84
Eastern BluebirdX
Yellow-rumped Warbler84
Song Sparrow21

You have observations from two trips in a single table. Trip date and locality are provided. Region is not provided. The wizard will request the name of a region after column types have been assigned. Date and locality are provided only when they change. Note that Gladys heard a Bluebird in Pinetops and that Mort did not. We will assume that the data are imported from a table in a web page.

  1. Press the button "Import trips and ticks" to obtain a wizard.
  2. Provide the web page.
    1. If the web page exists as a file on your computer, it's easiest to click the button "Browse files ..." and use the browser to locate the file.
    2. Click "Forward" to select the table type.
  3. Select the appropriate table type.
    1. In this example, there is one column for each observer, and each row represents one species that was seen on one trip.
    2. Select the bottom table type that is one column for each observer.
    3. Press "Forward" to view the web page and select the table.
  4. Select the portion of the table that is to be imported.
    1. Click on the date "2004-10-4" in the upper left-hand corner of the table. It should turn green.
    2. Click on the "1" in the lower right-hand corner of the table. The body of the table that lies below the header should turn green.
    3. Press "Forward" to extract data from the table and identify column types.
  5. Identify columns types.
    1. For each column click on the header and select the column type on the menu that pops up.
    2. Click "Forward" to provide the name of the region that will be used for both trips.
  6. Provide values for missing columns.
    1. Region was not provided in the table and is needed to specify trips.
    2. Replace the default region name provided in the table with an appropriate name.
    3. Press "Forward" to begin importing ticks.
  7. Be ready to identify species for which names in the table do not match those in the master species list.
  8. When the import has finished, click the forward button to see a summary and then dismiss the wizard.

Previous (Trips and ticks in detail) Contents Next (Checklists in detail)

SourceForge Logo

Last updated 18 March 2007