Collector Workflow: Offline-First Data Collection

Status: Final Version: 1.0 Last Updated: 2026-01-03

---

Purpose

Define the end-to-end workflow for field collectors to gather ESG data using mobile devices with offline-first architecture, including draft storage, queue synchronization, and feedback loops.

---

Workflow Diagram

[Mobile App] Draft → Queue → Upload → [Backend] Receive → Validate → Feedback
     ↓                                                                    ↓
  Auto-save                                                      Collector reviews
     ↓                                                                    ↓
  Local DB                                                          Resubmit/Fix

---

States & Transitions

State	Location	Description	User Actions	System Actions
Draft	Mobile only	Collector editing data	Edit, save, delete	Auto-save to local DB
Queued	Mobile only	Ready for upload, pending network	Retry upload	Wait for connectivity
Uploading	In-flight	HTTP request in progress	None (wait)	POST to /api/v1/collector/submissions
Received	Backend	Successfully ingested	None	Store in DB, queue validation job
Validated/Rejected	Backend	Validation complete	View feedback, fix, resubmit	Send sync response

---

Step-by-Step Workflow

Step 1: Fetch Templates (Online)

Trigger: Collector opens app with network connection.

Action:

GET /api/v1/collector/templates?reporting_period_id=10&site_id=123
Authorization: Bearer {token}

Mobile App Logic:

// Pseudo-code (Flutter/Dart)
Future<void> fetchTemplates() async {
  final templates = await api.getTemplates(periodId: currentPeriod.id, siteId: currentSite.id);

  // Store templates in local DB for offline access
  await db.templates.insertAll(templates);

  // Pre-create draft submissions for each required metric
  for (var metric in templates.metrics.where((m) => m.isMandatory)) {
    await db.submissions.insert(Submission(
      uuid: Uuid().v4(),
      metricId: metric.metricId,
      state: 'draft',
      value: null,
    ));
  }
}

---

Step 2: Create/Edit Draft (Offline-Safe)

Trigger: Collector enters data in form.

Mobile App Logic:

Future<void> saveDraft(Submission draft) async {
  // Client-side validation
  final errors = validateLocally(draft);
  if (errors.isNotEmpty) {
    showErrors(errors);
    return;
  }

  // Auto-save to local DB (SQLite)
  await db.submissions.upsert(draft.copyWith(
    state: 'draft',
    lastModified: DateTime.now(),
  ));

  showSnackbar('Draft saved');
}

Local Validation: - Schema validation (data type, required fields) - Domain validation (min/max, regex) - No backend call required

---

Step 3: Queue for Upload

Trigger: Collector taps "Submit" button.

Mobile App Logic:

Future<void> queueSubmission(String uuid) async {
  final draft = await db.submissions.getByUuid(uuid);

  // Final client-side validation
  final errors = validateLocally(draft);
  if (errors.isNotEmpty) {
    showErrors(errors);
    return;
  }

  // Mark as queued
  await db.submissions.update(uuid, {
    'state': 'queued',
    'queued_at': DateTime.now(),
  });

  // Trigger immediate upload if online
  if (await connectivity.isOnline()) {
    await uploadQueue();
  } else {
    showSnackbar('Queued for upload when online');
  }
}

---

Step 4: Upload to Backend

Trigger: Network connectivity detected or manual "Sync" button.

Mobile App Logic:

Future<void> uploadQueue() async {
  final queued = await db.submissions.where((s) => s.state == 'queued').get();

  for (var submission in queued) {
    try {
      await api.submitData(
        submissionUuid: submission.uuid,
        reportingPeriodId: submission.periodId,
        siteId: submission.siteId,
        metricId: submission.metricId,
        activityDate: submission.activityDate,
        value: submission.value,
        unit: submission.unit,
        idempotencyKey: submission.uuid, // UUID = idempotency key
      );

      // Mark as uploaded
      await db.submissions.update(submission.uuid, {
        'state': 'uploaded',
        'uploaded_at': DateTime.now(),
      });

    } on NetworkException {
      // Retry later (keep in queued state)
      break;
    } on ApiException catch (e) {
      // Server rejected (validation error)
      await db.submissions.update(submission.uuid, {
        'state': 'error',
        'error_message': e.message,
        'error_details': e.details,
      });
    }
  }
}

Backend Laravel Controller:

public function submit(SubmissionRequest $request)
{
    // Idempotency check
    $existing = MetricSubmission::where('submission_uuid', $request->submission_uuid)->first();
    if ($existing) {
        return response()->json($existing, 200); // Return existing, don't duplicate
    }

    $submission = MetricSubmission::create([
        'tenant_id' => auth()->user()->tenant_id,
        'submission_uuid' => $request->submission_uuid,
        'reporting_period_id' => $request->reporting_period_id,
        'site_id' => $request->site_id,
        'metric_definition_id' => MetricDefinition::where('metric_id', $request->metric_id)->value('id'),
        'raw_data' => $request->all(),
        'state' => 'RECEIVED',
        'submitted_by_user_id' => auth()->id(),
        'submitted_at' => now(),
    ]);

    // Queue async validation
    dispatch(new ValidateSubmissionJob($submission->id));

    return response()->json($submission, 201);
}

---

Step 5: Upload Evidence Files

Trigger: Collector attaches photos/PDFs to submission.

Mobile App Logic:

Future<void> uploadEvidence(String submissionUuid, File file, String evidenceType) async {
  final response = await api.uploadEvidence(
    submissionUuid: submissionUuid,
    file: file,
    evidenceType: evidenceType,
  );

  await db.evidence.insert(Evidence(
    id: response.evidenceId,
    submissionUuid: submissionUuid,
    filename: response.filename,
    uploaded: true,
  ));

  showSnackbar('Evidence uploaded');
}

---

Step 6: Sync Status & Feedback

Trigger: Periodic background sync (every 15 min) or manual refresh.

Mobile App Logic:

Future<void> syncStatus() async {
  final lastSync = await db.settings.get('last_sync_timestamp');

  final response = await api.sync(since: lastSync);

  for (var update in response.data) {
    await db.submissions.update(update.submissionUuid, {
      'state': update.state,
      'validation_errors': update.validationErrors,
      'reviewer_feedback': update.reviewerFeedback,
    });

    // Show notification if rejected
    if (update.state == 'REJECTED') {
      showNotification('Submission rejected: ${update.reviewerFeedback}');
    }
  }

  await db.settings.set('last_sync_timestamp', response.syncTimestamp);
}

---

Step 7: Resubmit After Rejection

Trigger: Collector fixes data based on feedback.

Mobile App Logic:

Future<void> resubmit(String originalUuid) async {
  final original = await db.submissions.getByUuid(originalUuid);

  // Create new submission (new UUID, version++)
  final newSubmission = original.copyWith(
    uuid: Uuid().v4(),
    state: 'draft',
    version: original.version + 1,
    supersedes: originalUuid,
  );

  await db.submissions.insert(newSubmission);

  showSnackbar('Ready to edit and resubmit');
}

Backend: New submission with metadata.supersedes = original_uuid links versions.

---

Error Handling

Network Failures

• Retry Logic: Exponential backoff (2s, 4s, 8s, 16s)
• Max Retries: 5 attempts
• User Notification: "Upload failed, will retry automatically"

Validation Errors (422)

• Move to 'error' state
• Display field-level errors to collector
• Allow editing and resubmission

Idempotency Violations (409)

• Backend returns existing submission
• Mobile app updates local record with server state
• No duplicate created

---

Laravel Jobs

ValidateSubmissionJob

class ValidateSubmissionJob implements ShouldQueue
{
    public $queue = 'validations';
    public $tries = 3;

    public function handle()
    {
        $submission = MetricSubmission::find($this->submissionId);

        $validator = app(ValidationService::class);
        $result = $validator->validate($submission);

        if ($result->passed()) {
            $submission->update(['state' => 'VALIDATED']);
            dispatch(new ProcessSubmissionJob($submission->id));
        } else {
            $submission->update([
                'state' => 'REJECTED',
                'validation_errors' => $result->errors(),
            ]);
        }
    }
}

---

Acceptance Criteria

• Mobile app works fully offline (draft creation, editing)
• Submissions queued locally and uploaded when online
• Idempotency prevents duplicate submissions
• Validation errors synced back to mobile with field-level details
• Evidence upload supports photos (JPEG/PNG) and PDFs
• Resubmission creates new version linked to original
• Background sync updates submission states every 15 minutes

---

Cross-References

• Collector API
• Ingestion Pipeline
• Validation Engine

---

Change Log

Version	Date	Author	Changes
1.0	2026-01-03	Senior Product Architect	Initial collector workflow specification