StereoMorph Digitizing Application Help

Contents

Last updated: February 6, 2015

Opening the App

Using the App

R code examples in this help file are taken from the StereoMorph Tutorial. In order to run these examples, download the StereoMorph Tutorial folder, make this folder the current working directory in R and loaded the StereoMorph library. For more information, please see Digitizing with StereoMorph.

To top

Uploading an image to the app

One or multiple images can be uploaded to the app. If multiple images are uploaded, the user can move back and forth between images within the app by clicking the "Previous Image" or "Next Image" buttons in the control panel. To upload a single image, input a single file path via the image.file parameter

> digitizeImage(
   image.file = "Object images/obj1_a1_v1.JPG",
   landmarks.file = "Landmarks 2D/obj1_a1_v1.txt",
   landmarks.ref = "landmarks_ref.txt")

where Object images/obj1_a1_v1.JPG is an image in the current working directory and landmarks_ref is a file with a list of landmarks to be digitized. To upload more than one image, either input a vector of file paths into image.file

> digitizeImage(
   image.file = c("Object images/obj1_a1_v1.JPG", "Object images/obj1_a1_v2.JPG"),
   landmarks.file = c("Landmarks 2D/obj1_a1_v1.txt", "Landmarks 2D/obj1_a1_v2.txt"),
   landmarks.ref = "landmarks_ref.txt")

or input a path to a folder containing the images ("Object images" below).

> digitizeImage(
   image.file = "Object images",
   landmarks.file = "Landmarks 2D",
   landmarks.ref = "landmarks_ref.txt")

Note that in this last case, the names of landmark files will be made the same as the image names, only with the extension '.txt'.

Different browsers will behave differently when switching between images. In Safari, pressing "Previous Image" or "Next Image" will cause the new image to appear in the same window (essentially behaving like a refresh event). In Chrome and Opera, switching to a different image would ordinarily cause the new window to open in a separate tab, meaning that the user would have to close the previous tab every time a new image was opened. To save the user from this additional step, switching images in Chrome and Opera is accomplished by closing the current window (other open tabs and windows will not be closed) and opening the new image in a new window. In this way, there is ultimately no difference in performance among Safari, Chrome and Opera. Firefox, however, does not allow web script to close the current window. This means that Firefox users will have one additional step of closing the previous window when switching between images.

To top

Uploading a list of landmarks to be digitized

A list of landmarks or points to be digitized is specified when calling digitizeImage(). There are two ways of doing this. The file path of a .txt file containing a list of the landmarks to be digitized (landmarks_ref.txt below) can be input via the landmarks.ref parameter.

> digitizeImage(
   image.file = "Object images",
   landmarks.file = "Landmarks 2D",
   landmarks.ref = "landmarks_ref.txt")

The landmarks in landmarks_ref.txt should be listed in a single column, each on a separate line.

cranium_occipital
basioccipital_proc_ant
basipterygoid_proc_ant_L
basipterygoid_proc_post_L
...

Alternatively, the landmarks can be input as a vector.

> digitizeImage(
   image.file = "Object images",
   landmarks.file = "Landmarks 2D",
   landmarks.ref = c("cranium_occipital", "foramen_magnum_sup", "upperbeak_tip"))

To top

Uploading a list of curves to be digitized

A list of curves to be digitized is specified when calling digitizeImage(). There are two ways of doing this. The file path of a .txt file containing a list of the curves to be digitized (curves_ref.txt below) can be input via the curves.ref parameter.

> digitizeImage(
   image.file = "Object images/obj1_a2_v1.JPG",
   control.points.file = "Control points/obj1_a2_v1.txt",
   curve.points.file = "Curve points 2D/obj1_a2_v1.txt",
   curves.ref = "curves_ref.txt")

The StereoMorph Digitizing App assumes that all curves will be bound at each end by a landmark. These are input into digitizeImage() as a three-column matrix.

tomium_L             upperbeak_tip            upperbeak_tomium_prox_L
tomium_R             upperbeak_tip            upperbeak_tomium_prox_R
pterygoid_crest_L    pterygoid_crest_ant_L    pterygoid_crest_post_L
...

Where the first column is the curve name, the second column is the starting landmark and the third column is the end landmark. The app will treat each curve start and end point as both landmarks and curve control points. So if you move a curve start and end point, its position will also be updated in the landmark list (if a landmark file is also being saved). In this way, landmarks that also define the start and end of curves do not have to be digitized twice.

Alternatively, curves can also be input as a matrix directly

> digitizeImage(
   image.file = "Object images/obj1_a2_v1.JPG",
   control.points.file = "Control points/obj1_a2_v1.txt",
   curve.points.file = "Curve points 2D/obj1_a2_v1.txt",
   curves.ref = matrix(
      c("tomium_L", "upperbeak_tip", "upperbeak_tomium_prox_L",
      "tomium_R", "upperbeak_tip", "upperbeak_tomium_prox_R"),
   nrow=2, ncol=3, byrow=TRUE))

The landmark names used in the curve reference file can also be included in the landmark reference file but do not have to be. Landmark names in the curve reference file not present in the landmark reference file will be added automatically.

To top

Uploading and saving to landmark files

The file path or paths at which landmarks will be saved is specified when calling digitizeImage(). There are two ways of doing this. The file paths of each landmark file can be input as a vector. In this case the number of landmark filenames must be equal to the number of image filenames.

> digitizeImage(
   image.file = c("Object images/obj1_a1_v1.JPG", "Object images/obj1_a1_v2.JPG"),
   landmarks.file = c("Landmarks 2D/obj1_a1_v1.txt", "Landmarks 2D/obj1_a1_v2.txt"),
   landmarks.ref = "landmarks_ref.txt")

Alternatively, a single folder can be input to landmarks.file.

> digitizeImage(
   image.file = "Object images",
   landmarks.file = "Landmarks 2D",
   landmarks.ref = "landmarks_ref.txt")

In this last case, the names of landmark files will be copied over from the names of the images (whether the image names are input as a vector or folder), replacing the image extension (.jpg, .tiff, etc.) with ".txt".

To top

Uploading and saving to curve files

Bézier curves and splines are defined by control points. These are the points in the image frame that can be moved around with the mouse, causing the curve to change shape. When digitizing curves, these points must be saved; they are used by the App to display the curves in the image frame and can easily be re-loaded at any point if the curves need to be modified. The file path or paths at which control points will be saved is input via the control.points.file parameter.

However, the control points are not suitable for comparisons among objects or for 3D reconstruction since an infinite number of control point configurations can define the same Bézier curve. Thus, the StereoMorph Digitizing App can also generate points at single-pixel spacing along the curve (see below), for use in 3D reconstruction and subsequent analyses (these are not used by the app in any way).

pterygoid_crest_R0001   1140   1672
pterygoid_crest_R0002   1140   1671
pterygoid_crest_R0003   1140   1670
pterygoid_crest_R0004   1140   1669
...

The file path or paths at which curve points will be saved is input via the curve.points.file parameter.

To save both the control points and curve points, enter file paths or folders for both control.points.file and curve.points.file.

> digitizeImage(
   image.file = "Object images/obj1_a2_v1.JPG",
   control.points.file = "Control points/obj1_a2_v1.txt",
   curve.points.file = "Curve points 2D/obj1_a2_v1.txt",
   curves.ref = "curves_ref.txt")

To only save the control points, leave curve.points.file as NULL (default).

> digitizeImage(
   image.file = "Object images/obj1_a2_v1.JPG",
   control.points.file = "Control points/obj1_a2_v1.txt",
   curves.ref = "curves_ref.txt")

To top

Uploading and saving landmarks and curves

To upload and save landmarks and curves, include all of the input parameters for landmarks and curves.

> digitizeImage(
   image.file = "Object images",
   landmarks.file = "Landmarks 2D",
   control.points.file = "Control points",
   curve.points.file = "Curve points 2D",
   landmarks.ref = "landmarks_ref.txt",
   curves.ref = "curves_ref.txt")

This will display both landmarks and curves with curve start and end points being updated simultaneously with their corresponding landmarks. To save landmarks, control points and curve points, click "Submit All" in the control panel.

To top

Image compatibilities

Images can only be digitized if the image type is compatible with the browser being used (the user's default browser). JPEG and PNG are widely supported image types - they will load in all the browsers with which the digitizing application is compatible. TIFF images can only be digitized in Safari.

To top

Keyboard/mouse shortcuts

x Add a marker, re-position a marker, select/unselect marker
n Select next marker
p Select previous marker
d Delete a selected marker
shift + a Turn on/off auto-advance
double-click Add a marker, re-position a marker, select/unselect marker
scroll over image Zoom in/out
click-and-drag Move image or marker
arrow Move image or marker in one pixel increments up/down/right/left
shift + arrow Move image or marker in 10 pixel increments up/down/right/left
shift + cmnd + arrow Move image or marker in 100 pixel increments up/down/right/left
refresh Restore original landmarks/curves from when app was initially loaded (does not change saved files)
To top

Zooming in and out of the image

Zoom in and out of an image using the scroll feature of a trackpad or mouse scroll wheel while your cursor is positioned somewhere over the image (as with Google Maps). To more quickly navigate around the image, the zoom tracks the position of your cursor and zooms in and out of particular region of the image based on the current position of your cursor. For instance if you wanted to zoom in to the bottom-left corner of the image, position your cursor in the bottom-left of the image and scroll in the appropriate direction. This will increase the size of the image while simultaneously positioning the bottom-left corner of the image into the center of the image frame. The "Current zoom" field in the bottom right of the window displays the current zoom of the image. Images will be fit to the window on loading and so the initial zoom will depend on the dimensions of the image.

To top

Moving around the image

Once you have zoomed into the image, you can move the image simply by clicking-and-dragging somewhere on the image (as with Google Maps). This will cause the image to move with your cursor. The image will not continue to move with the cursor once its border is flush with the border of the image frame. Additionally, the image will not move if you are near or on top of a selected marker. In this case, clicking-and-dragging will move the marker rather than the image (see Moving landmarks).

You can also move the image using the up, right, down and left arrow keys. In this case make sure that no landmarks or control points are selected or else the selected point will be moved instead of the image (see Selecting and unselecting landmarks). The image can be moved in increments of one pixel (arrow keys only), 10 pixels (arrow keys while holding down shift), or 100 pixels (arrow keys while holding down shift and command).

To top

Digitizing landmarks

To digitize a landmark, first select the landmark you want to digitize from the right side panel. Click on the "Landmarks" tab to show the list of loaded landmarks. Select a landmark by clicking anywhere in the row corresponding to that landmark. The text in the row should turn bold and the name of the landmark should appear in the "Current object" field in the bottom right of the window. Then, with your cursor positioned over the image, double-click or press 'x'. Note that if your cursor is positioned near or over another landmark, this will cause that landmark to be selected rather than creating a new one. If auto-advance is on, once you add a new landmark it will automatically advance to the next landmark. You can turn auto-advance on and off by holding down shift and pressing 'a' or (initially) via the input parameters to digitizeImage().

To top

Selecting and unselecting landmarks

Before changes are made to a landmark, it must first be selected. If the landmark has not yet been digitized, select it by clicking on the corresponding row in the right "Landmarks" panel. If the landmark has been digitized, you can either double-click the landmark on the image or press 'x' with the cursor over the landmark. The text in the corresponding row of the landmarks panel should turn bold, the name of the landmark should appear in the "Current object" field and (if digitized) the landmark on the image will change to the specified landmark.focus.color (green by default). To unselect a landmark, simply select it again by any of the same actions. This will cause the landmark to lose focus without selecting another marker.

To top

Moving landmarks

You can move already digitized landmarks either using the mouse or keyboard keys. First, select the landmark (by double-clicking the landmark on the image, pressing 'x' with the cursor over the landmark or selecting the corresponding row in the right "Landmarks" panel). The text in the corresponding row of the landmarks panel should turn bold, the name of the landmark should appear in the "Current object" field and the landmark on the image will change to the specified landmark.focus.color (green by default). You can use the mouse to click-and-drag the landmark to a new location or move the cursor to the new desired location and double-click or press 'x'.

You can also move the landmark using the up, right, down and left arrow keys. Points can be moved in increments of one pixel (arrow keys only), 10 pixels (arrow keys while holding down shift), or 100 pixels (arrow keys while holding down shift and command). If auto-advance is on, double-clicking or pressing 'x' will cause the app to automatically advance to the next landmark. Clicking-and-dragging or movement with the arrow keys will not trigger auto-advance.

To top

Deleting landmarks

To remove an already digitized landmark, first select the point (by double-clicking the landmark on the image, pressing 'x' with the cursor over the landmark or selecting the corresponding row in the right "Landmarks" panel). The text in the corresponding row of the Landmarks panel should turn bold, the name of the landmark should appear in the "Current object" field and the landmark on the image will change to the specified landmark.focus.color (green by default). Then, press the letter 'd' (for delete). This will cause the x and y coordinates in the right Landmarks panel to change to '-'; landmarks whose coordinates are '-' will not be saved when submitting landmarks. If you wish to remove the landmark entirely from the landmark list, remove the landmark from the reference landmarks input to digitizeImage().

To top

Advancing through landmarks

To quickly move among landmarks use the 'n' and 'p' keys. 'n' will advance to the next landmark (in the order given in landmarks.ref and shown in the Landmarks panel) and 'p' will move backward to the previous landmark.

To top

Digitizing curves

The StereoMorph Digitizing Application currently only allows for collection of curves in the form of a Bézier. A Bézier curve is defined by a series of control points; points on either end of the curve are points through which the curve must pass while a point in the middle pulls the curve in a particular direction without the curve actually passing through it. The Bézier curves in the StereoMorph Digitizing Application are Bézier splines, composed of one or more Quadratic Bézier curves. That is, they are a series of Bézier curves with two end points and one middle point joined together into one continuous curve. Thus, unless you have the correct number of points required for a Bézier curve (three) or spline (2n + 1, where n is the number of Bézier curves in the spline) the entire Bézier curve will not be drawn. This will be evident as an absence of a connecting line between adjacent points. Simply add another point until the curve is complete.

The StereoMorph Digitizing Application assumes that all curves will be bound at each end by a landmark. In the right "Curves" side panel each curve is listed as its own block first with the name of the curve and then the start and end points (which are also treated as landmarks). The fields in the middle without labels are where the middle control points will be added. The ends of each curve (specified in the curves.ref matrix) will be added to the landmark list (if they are not already included) and the landmark listing will be updated along with any changes made to the curve ends.

It is easiest to digitize a Bézier curve by first specifying the start and end points. Begin by selecting the starting point of the curve by clicking on the label or coordinate fields corresponding to the starting point. The text in the row should turn bold and the name of the curve starting point should appear in the "Current object" field in the bottom right of the window. Then, just as with landmarks, position your cursor over the image and double-click or press 'x'. Note that if your cursor is positioned near or over another landmark or control point, this will cause that marker to be selected rather than creating a new one. If auto-advance is on, the app will automatically advance to (select) the end point of the curve. If auto-advance is off, select the end point of the curve manually or press 'n' (for next). Digitize the end point as with the starting point. Next, select the middle control point by pressing 'n' again or clicking on either of the '-' fields in the Curves panel between the start and end curve points. This will be selected automatically if auto-advance is on. Digitizing the middle control point will cause the curve to be drawn in. Continue adding and moving control points (see Modifying Curves) until the Bézier curve/spline has the desired shape. Note that since the Bézier spline consists of a concatenated string of quadratic Bézier curves, spline points alternate between end and directional points.

To top

Modifying curves

Curves can be modified just as with landmarks. First, select the desired control point (either start, middle or end) by double-clicking the point on the image, pressing 'x' with the cursor over the point or selecting the corresponding row in the "Curves" panel). The text in the corresponding row of the Curves panel should turn bold, the name of the control point should appear in the "Current object" field and the control point on the image will change to the specified control.point.focus.color (red by default). You can use the mouse to click-and-drag the control point to a new location or move the cursor to the new desired location and double-click or press 'x'.

You can also move the point using the up, right, down and left arrow keys. Points can be moved in increments of one pixel (arrow keys only), 10 pixels (arrow keys while holding down shift), or 100 pixels (arrow keys while holding down shift and command). If auto-advance is on, double-clicking or pressing 'x' will cause the app to automatically advance to the next control point. Clicking-and-dragging or movement with the arrow keys will not trigger auto-advance.

Control points can be deleted in the same way as landmarks. First select the point (by double-clicking the point on the image, pressing 'x' with the cursor over the point or selecting the corresponding row in the "Curves" panel). Then, press the letter 'd'. This will cause the x and y coordinates in the Curves panel to change to '-'; control points whose coordinates are '-' will not be saved when submitting curves or used in determining the pixel-by-pixel curve points.

To top

Deleting curves

To delete an entire curve simply delete all the points used to define the curve (start, middle and end). First select the starting point (by double-clicking the point on the image, pressing 'x' with the cursor over the point or selecting the corresponding row in the "Curves" panel). Then, press the letter 'd' (for delete). Select the next control point by pressing 'n' and, again, press 'd' to delete this point. Continue moving through and deleting each point of the curve using the 'n' and 'd' keys. This will cause the x and y coordinates in the right Curves panel to change to '-'; curves with all '-' values will not be saved when submitting curves. Curves with only a start and end point will not be saved either when submitting curves, however, these points will be saved as landmarks if "Submit Landmarks" is pressed. If you wish to remove the curve entirely from the curve list, remove the curve from the reference curves input to digitizeImage().

To top

Control points versus curve points

The points that are digitized in this application and whose coordinates appear in the right Curves panel are Bézier control points. These are used to generate a Bézier curve or spline which can then be used in further analyses. The control points cannot generally be analyzed directly because an infinite number of control point configurations can define the same Bézier curve. To streamline the data collection process, the StereoMorph Digitizing Application can generate curve points that can be used in subsequent analyses. This is done by calling digitizeImage() and including a file path at which to save the curve points (curve.points.file). The curve points are generated at single pixel spacing using the R package bezier. These single-pixel spaced curve points are generated during the saving process and are not used at any point by the digitizing application. Curves are loaded into and modified in the application as control points. If curve.points.file is not input into digitizeImage() the single-pixel spaced curve points are not generated and only the control points are saved.

To top

Copying landmarks and curves between images

When digitizing a series of images with landmarks or curves in similar positions from one image to the next, it is often useful to copy landmarks and curves digitized in one image to the next image. In this way, landmarks and curve points need only be adjusted slightly in subsequent images. To do this, click on the "Settings" tab and check the box next to "Copy landmarks to next image" or "Copy curves to next image". The next image refers to the image loaded after the current image, regardless of whether the image is reached by clicking "Previous Image" or "Next Image". Landmarks and curves will only be copied to the subsequent image if they are saved for the current image and if landmarks and curves are not already digitized for the subsequent image. Additionally, any copied landmarks or curves are not automatically saved for the image on to which they are copied; they must be saved by clicking "Submit". A warning message will appear if users try to leave the current image without submitting copied landmarks and curves.

To top

Resizing the browser window

The browser window can be resized at any time during digitizing. The image will be re-sized accordingly, however the zoom will be reset.

To top

Turning on/off auto-advance

Auto-advance can greatly increase the speed at which points can be digitized, especially when the points are repetitive and at nearly regular spacing. With auto-advance on, once you digitize a landmark or control point, the next landmark or control point in sequence is automatically selected. The order in which landmarks are advanced is the same as the order in which they were entered in landmarks.ref. Thus, the input order can be made to match the distribution of points on the object to advance through the landmarks most efficiently. To turn auto-advance on and off, mark the auto.advance parameter as either TRUE or FALSE when calling digitizeImage(). Once the application has loaded, auto-advance can be turned on and off by holding down the shift key and pressing 'a'.

To top

Moving between images

If multiple images have been uploaded to digitizeImage() (see Uploading an image or images to the app) the user can move back and forth between images by clicking the "Previous Image" or "Next Image" buttons in the control panel.

Different browsers will behave differently when switching between images. In Safari, pressing "Previous Image" or "Next Image" will cause the new image to appear in the same window (essentially behaving like a refresh event). In Chrome and Opera, switching to a different image would ordinarily cause the new window to open in a separate tab, meaning that the user would have to close the previous tab every time a new image was opened. To save the user from this additional step, switching images in Chrome and Opera is accomplished by closing the current window (other open tabs and windows will not be closed) and opening the new image in a new window. In this way, there is ultimately no difference in performance among Safari, Chrome and Opera. Firefox, however, does not allow web script to close the current window. This means that Firefox users will have one additional step of closing the previous window when switching between images.

To top

Browser compatibility

The StereoMorph Digitizing App runs in the user's default browser. Safari, Chrome and Opera provide full compatibility with all features of the app, with only slight differences in behavior (such as when moving between images). Firefox provides almost complete compatibility, with the exception of automated closing of windows. When switching between images and in exiting the application, Firefox users must manually close the previous or current window. The app has not been tested with Internet Explorer as IE has compatibility issues in websocket communication between the browser and R.