Documentation

1. Overview

RealityExtract is an Android application for automated optical character recognition (OCR) of tagged displays and instruments. It uses hashtag markers (e.g., #1#, #2#) to identify measurement points and continuously captures their numeric values.

Key Features

• Tag-based identification - Place hashtag markers next to displays for automatic detection
• On-device OCR - Deep learning-based engine runs entirely on-device, no network required
• Auto-correction - Automatically finds optimal preprocessing settings for each tag
• Configurable sampling - Set capture intervals from 500ms to several seconds
• CSV export - Export captured data with timestamps for analysis
• Debug capture - Diagnostic data for troubleshooting OCR issues

2. Quick Start

1. Install the app - Download and install the APK from the
2. Create a project - Open the app and create a new project
3. Place tag markers - Print or write hashtag markers (e.g., #1#) and place them next to the displays you want to capture
4. Scan for tags - From the project screen, tap "Scan for Tags". The app detects visible tags and guides you through configuring each tag's properties (name, value range, LED format)
5. Start capture - Tap "Start Capture". The app calibrates tag positions, then continuously reads and records values
6. Export - Export the captured data as CSV for analysis

3. Projects & Sessions

Projects

A project represents a specific setup or measurement configuration. Each project stores:

• Tag mappings with per-tag properties (value range, LED display format)
• Sampling interval
• Preprocessing settings
• Calibration data (tag positions and value regions)
• Auto-correction results

Sessions

A session is a single capture run within a project. Each session contains:

• Timestamped readings for each tag
• Start and end times
• Statistics (total readings, null rates, etc.)
• Optional debug data

4. Calibration Process

Calibration is the process of detecting tag markers and determining where to read values from. It consists of several phases:

Calibration Phases

Value Region Detection

For each detected tag, the app determines where the numeric value is located. This can happen in two ways:

• Automatic detection - The OCR engine detects numeric text near the tag and uses that bounding box as the value region
• Fallback region - If no value is detected during calibration, a default region is calculated relative to the tag position (approximately 1.5x tag width, 2.5x tag height)

5. Auto-Correction

Auto-correction automatically finds the best preprocessing settings for each tag's value region. This is particularly useful when different displays require different image processing to be read reliably.

How It Works

1. After tag detection, the app enters a validation phase
2. For each tag, it attempts to read the value using the current preprocessing preset
3. After 5 attempts, if the success rate is below 70%, it moves to the next preset
4. If the success rate is above 90%, it immediately selects that preset
5. After testing all presets (or reaching the 120-second timeout), it selects the best-performing preset
6. The selected preset is stored as the tag's preprocessing override

Preset Cycling Order

The app first tries with no preprocessing (default), then cycles through these presets in order. LED/inverted presets are tested first since they're most effective for digital displays:

Expected Value Range (Min/Max)

You can optionally set minimum and maximum expected values for each tag. Values outside this range are rejected both during capture and during calibration validation.

How It Works

1. Set min and max values for a tag in the tag properties (tap a tag to edit, e.g., min=0, max=100 for a percentage display)
2. During capture, OCR readings outside the range are rejected and not recorded
3. During calibration validation, out-of-range readings are counted as failures for preset selection
4. This helps filter out misreadings (e.g., "1234" when the display shows "12.34")

When to Use

• When you know the expected value range (e.g., 0-100 for percentages, 0-50 for temperatures)
• When decimal points are hard to detect (LED displays often show "2345" instead of "23.45")
• To reject obviously incorrect readings during capture

6. Preprocessing Settings

Preprocessing uses OpenCV to enhance images before OCR. All preprocessing happens on-device.

Available Settings

Processing Pipeline

When preprocessing is enabled, the following steps are applied in order:

1. Convert to grayscale
2. Apply CLAHE (if enabled)
3. Apply binary thresholding (if enabled)
4. Invert colors (if enabled)
5. Send to OCR engine

7. OCR Modes

The app uses a deep learning-based recognition engine that runs entirely on-device, with specialized preprocessing for different display types.

On-Device OCR Engine

General-purpose OCR that works well with most printed text and digital displays. Uses a deep learning model trained on diverse text styles. The engine includes:

• Text detection - Locates text regions in the image
• Text recognition - Converts detected regions to text
• Preprocessing pipeline - Applies CLAHE, thresholding, and LED mode as needed

LED/LCD Display Support

For LED and LCD displays (digital scales, meters, industrial equipment), use the LED mode preprocessing option. This extracts bright pixels on dark backgrounds and applies specialized filtering for segment-style displays.

8. Debug Data Capture

When enabled, debug data capture collects diagnostic information to help troubleshoot OCR issues. This data is uploaded to the cloud and can be viewed in the admin dashboard.

What's Captured

• Calibration images - Raw and annotated frames showing detected tags
• Capture images - Sample frames during capture with overlay regions
• Cropped regions - Before/after preprocessing images for each tag
• Device logs - Logcat output filtered to app-specific tags
• Session metadata - Frame dimensions, timing, tag metrics, issues

Viewing Debug Data

Debug sessions can be viewed in the admin dashboard at /admin/debug. Each session shows:

• Session summary and status
• Detected issues (null values, low confidence, dimension mismatches)
• Per-tag statistics and auto-correction results
• Image samples with before/after preprocessing comparison
• Filterable device logs

9. Auto-Stop Timer

The auto-stop timer lets you set a maximum duration for a capture session. When the timer expires, the capture stops automatically. This is useful for unattended recordings or when you know exactly how long you need to capture data.

How to Use

1. On the project screen, locate the duration slider above the "Start Capture" button
2. Slide to select a preset duration: 5 min, 10 min, 15 min, 30 min, 1h, 2h, or 4h
3. Leave the slider all the way to the left for "No limit" (manual stop only)
4. Start the capture — the top bar displays an "Auto-stop: Xm" or "Auto-stop: Xh" indicator

Pause & Resume

If you pause a timed capture, the remaining time is preserved. When you resume, the timer continues from where it left off rather than restarting.

10. Web Dashboard

The web dashboard at realityextract.com provides tools for viewing and analyzing captured data. Sign in with your account to access your projects and sessions.

Session View

Each session page displays captured readings with interactive chart and table views. The following features are available:

Data Processing

The processing panel provides algorithms to clean and smooth captured data:

• Interpolate NULL values using neighboring data points
• Correct outlier values that deviate significantly from neighbors
• Apply moving average smoothing to reduce noise
• Remove sudden spikes based on rate of change

11. Subscription & Plans

RealityExtract offers a free tier and a Pro subscription to fit different usage needs.

Available Plans

Top-Up Packs

Pro subscribers can purchase additional capture hours in top-up packs. Top-up hours never expire and stack on top of the monthly allowance.

How to Subscribe

• Web: From the web dashboard, go to the Subscription page to manage your plan and purchase top-ups via Stripe
• Android: From the Android app, subscription options are available through Google Play Billing

12. Troubleshooting