# Insert Menu Items Service - Implementation Guide

## Overview

The `insert_menu_items` service provides an asynchronous workflow for mobile applications to upload menu images, extract menu data, and automatically create menus in the external zavedenia.com system.

### Flow
1. Mobile app uploads image + city + token → `insert_menu_items` endpoint
2. Service returns immediate success response (202 Accepted)
3. Background process:
   - Calls `menu_extraction` service with the uploaded image
   - Transforms the response using an adapter
   - Calls external `createMenu.php` API with transformed data
4. Errors are logged, success is logged

## Architecture Components

### 1. Service Structure

Create the following directory structure:

```
app/services/insert_menu_items/
├── __init__.py
├── config.py              # Service configuration
├── routes.py              # FastAPI routes
├── service.py             # Core business logic
├── adapter.py             # Response transformer
└── models/
    ├── __init__.py
    ├── requests.py        # Request models
    └── responses.py       # Response models
```

### 2. Configuration File

**File:** `config/services/insert_menu_items.yaml`

```yaml
# Insert Menu Items Service Configuration
enabled: true

# Authentication
auth:
  bearer_token_env: "INSERT_MENU_ITEMS_SERVICE_KEY"

# External API Configuration
external_api:
  create_menu_url: "https://zavedenia.com/apps/orderapprestaurant/api/createMenu.php"
  timeout: 30.0
  max_retries: 3

# Temporary file storage
storage:
  temp_directory: "/tmp/menu_images"
  cleanup_after_seconds: 3600  # 1 hour
```

### 3. Models

**File:** `app/services/insert_menu_items/models/requests.py`

```python
from typing import Optional
from pydantic import BaseModel, Field


class InsertMenuItemsRequest(BaseModel):
    """Request model for insert_menu_items endpoint"""
    city: str = Field(..., description="City identifier (e.g., 'sofia')")
    token: str = Field(..., description="Authentication token for createMenu.php API")
    source_language: str = Field(default="bg", description="Source language of the menu")
    target_language: str = Field(default="en", description="Target language for translation")
```

**File:** `app/services/insert_menu_items/models/responses.py`

```python
from pydantic import BaseModel


class InsertMenuItemsResponse(BaseModel):
    """Immediate response for insert_menu_items endpoint"""
    status: str = "processing"
    message: str = "Image uploaded successfully. Menu extraction and creation in progress."
    request_id: str  # Unique identifier for tracking
```

### 4. Adapter

**File:** `app/services/insert_menu_items/adapter.py`

The adapter transforms `MenuExtractionResponse` to the format expected by `createMenu.php`.

```python
from typing import Dict, Any, List, Optional
import structlog
from decimal import Decimal

from ..menu_extraction.models.dto import MenuExtractionResponse, MenuItem, MenuCategory

logger = structlog.get_logger("insert_menu_items.adapter")


class CreateMenuAdapter:
    """Transforms MenuExtractionResponse to createMenu.php format"""

    @staticmethod
    def transform(
        menu_response: MenuExtractionResponse,
        city: str,
        token: str
    ) -> Dict[str, Any]:
        """
        Transform menu extraction response to createMenu.php request format

        Args:
            menu_response: Response from menu_extraction service
            city: City identifier
            token: API token for createMenu.php

        Returns:
            Dictionary matching createMenu.php expected format
        """
        logger.info(
            "adapter_transform_started",
            categories_count=len(menu_response.menu.categories),
            city=city
        )

        menu_items = []

        for category in menu_response.menu.categories:
            transformed_category = CreateMenuAdapter._transform_category(
                category,
                menu_response.source_language,
                menu_response.target_language
            )
            menu_items.append(transformed_category)

        result = {
            "city": city,
            "token": token,
            "menu": menu_items
        }

        logger.info(
            "adapter_transform_completed",
            categories_count=len(menu_items),
            city=city
        )

        return result

    @staticmethod
    def _transform_category(
        category: MenuCategory,
        source_lang: str,
        target_lang: str
    ) -> Dict[str, Any]:
        """Transform a single category"""

        return {
            "name": category.name,  # Source language name
            "name_uk": category.name if target_lang == "en" else category.name,  # Target language name
            "description": category.description or "",
            "description_uk": category.description or "",
            "items": [
                CreateMenuAdapter._transform_item(item, source_lang, target_lang)
                for item in category.items
            ]
        }

    @staticmethod
    def _transform_item(
        item: MenuItem,
        source_lang: str,
        target_lang: str
    ) -> Dict[str, Any]:
        """Transform a single menu item"""

        # Extract price value (convert string to float)
        price = CreateMenuAdapter._extract_price(item.price)

        # Extract serving size (gramaj)
        gramaj = CreateMenuAdapter._extract_gramaj(item.serving_size)

        return {
            "name": item.name,  # Source language
            "name_uk": item.name,  # Target language (from translation)
            "description": item.description or "",
            "description_uk": item.description or "",
            "measure": 1,  # Default measure type
            "items": [
                {
                    "name": "Стандартна порция",  # Standard portion
                    "gramaj": gramaj,
                    "cena": price
                }
            ]
        }

    @staticmethod
    def _extract_price(price_str: Optional[str]) -> float:
        """
        Extract numeric price from price string

        Examples:
            "15.00 лв" -> 15.0
            "€10" -> 10.0
            "20" -> 20.0
        """
        if not price_str:
            return 0.0

        try:
            # Remove currency symbols and extract numbers
            import re
            numbers = re.findall(r'\d+\.?\d*', price_str)
            if numbers:
                return float(numbers[0])
            return 0.0
        except Exception as e:
            logger.warning("price_extraction_failed", price_str=price_str, error=str(e))
            return 0.0

    @staticmethod
    def _extract_gramaj(serving_size: Optional[str]) -> int:
        """
        Extract gramaj (weight) from serving size string

        Examples:
            "100g" -> 100
            "250 гр" -> 250
            "1kg" -> 1000
        """
        if not serving_size:
            return 100  # Default 100g

        try:
            import re
            # Look for numbers followed by g, gr, kg, etc.
            match = re.search(r'(\d+)\s*(g|gr|kg|гр)', serving_size.lower())
            if match:
                value = int(match.group(1))
                unit = match.group(2)

                # Convert kg to grams
                if unit == 'kg':
                    value *= 1000

                return value

            # Try to extract just numbers
            numbers = re.findall(r'\d+', serving_size)
            if numbers:
                return int(numbers[0])

            return 100  # Default
        except Exception as e:
            logger.warning("gramaj_extraction_failed", serving_size=serving_size, error=str(e))
            return 100
```

### 5. Service Implementation

**File:** `app/services/insert_menu_items/service.py`

```python
import uuid
import tempfile
import os
from pathlib import Path
from typing import Optional
import httpx
import structlog

from ..menu_extraction.service import MenuExtractionService, ImageDocument
from .adapter import CreateMenuAdapter
from .config import InsertMenuItemsServiceConfig
from ...shared.exceptions.base import (
    MenuExtractionException,
    ExternalServiceException,
    ValidationException
)

logger = structlog.get_logger("insert_menu_items.service")


class InsertMenuItemsService:
    """Service for handling menu image uploads and menu creation"""

    def __init__(
        self,
        config: InsertMenuItemsServiceConfig,
        menu_extraction_service: MenuExtractionService
    ):
        self.config = config
        self.menu_extraction_service = menu_extraction_service
        self.adapter = CreateMenuAdapter()

    async def process_menu_upload(
        self,
        *,
        image_content: bytes,
        filename: str,
        mime_type: str,
        city: str,
        token: str,
        source_language: str = "bg",
        target_language: str = "en",
        request_id: str
    ) -> None:
        """
        Background task to process menu image and create menu

        This method runs asynchronously after returning response to client.
        Errors are logged but not raised.
        """
        logger.info(
            "menu_upload_processing_started",
            request_id=request_id,
            city=city,
            filename=filename,
            source_language=source_language,
            target_language=target_language
        )

        try:
            # Step 1: Extract menu from image using menu_extraction service
            logger.info("calling_menu_extraction_service", request_id=request_id)

            image_doc = ImageDocument(
                content=image_content,
                mime_type=mime_type,
                filename=filename
            )

            menu_response = await self.menu_extraction_service.extract_menu(
                images=[image_doc],
                source_language=source_language,
                target_language=target_language,
                api_key_override=None
            )

            logger.info(
                "menu_extraction_completed",
                request_id=request_id,
                categories_count=len(menu_response.menu.categories)
            )

            # Step 2: Transform response using adapter
            logger.info("transforming_menu_data", request_id=request_id)

            create_menu_payload = self.adapter.transform(
                menu_response=menu_response,
                city=city,
                token=token
            )

            logger.info(
                "menu_data_transformed",
                request_id=request_id,
                categories_count=len(create_menu_payload["menu"])
            )

            # Step 3: Call external createMenu.php API
            logger.info("calling_create_menu_api", request_id=request_id, city=city)

            await self._call_create_menu_api(create_menu_payload, request_id)

            logger.info(
                "menu_upload_processing_completed",
                request_id=request_id,
                city=city
            )

        except MenuExtractionException as e:
            logger.error(
                "menu_extraction_failed",
                request_id=request_id,
                error=str(e)
            )
        except ExternalServiceException as e:
            logger.error(
                "create_menu_api_failed",
                request_id=request_id,
                error=str(e)
            )
        except Exception as e:
            logger.error(
                "menu_upload_processing_failed",
                request_id=request_id,
                error=str(e),
                error_type=type(e).__name__
            )

    async def _call_create_menu_api(
        self,
        payload: dict,
        request_id: str
    ) -> None:
        """Call external createMenu.php API with retry logic"""

        url = self.config.create_menu_url
        timeout = self.config.timeout
        max_retries = self.config.max_retries

        async with httpx.AsyncClient() as client:
            for attempt in range(1, max_retries + 1):
                try:
                    logger.info(
                        "create_menu_api_request",
                        request_id=request_id,
                        attempt=attempt,
                        max_retries=max_retries,
                        url=url
                    )

                    response = await client.post(
                        url,
                        json=payload,
                        headers={"Content-Type": "application/json"},
                        timeout=timeout
                    )

                    response.raise_for_status()

                    logger.info(
                        "create_menu_api_success",
                        request_id=request_id,
                        status_code=response.status_code,
                        response_body=response.text[:200]  # Log first 200 chars
                    )

                    return

                except httpx.HTTPStatusError as e:
                    logger.error(
                        "create_menu_api_http_error",
                        request_id=request_id,
                        status_code=e.response.status_code,
                        response_body=e.response.text[:200],
                        attempt=attempt
                    )

                    if attempt == max_retries:
                        raise ExternalServiceException(
                            "createMenu.php",
                            f"HTTP {e.response.status_code}: {e.response.text[:100]}"
                        )

                except httpx.RequestError as e:
                    logger.error(
                        "create_menu_api_request_error",
                        request_id=request_id,
                        error=str(e),
                        attempt=attempt
                    )

                    if attempt == max_retries:
                        raise ExternalServiceException(
                            "createMenu.php",
                            f"Request failed: {str(e)}"
                        )

                # Wait before retry (exponential backoff)
                if attempt < max_retries:
                    import asyncio
                    wait_time = 2 ** attempt  # 2, 4, 8 seconds
                    logger.info(
                        "create_menu_api_retry_wait",
                        request_id=request_id,
                        wait_seconds=wait_time
                    )
                    await asyncio.sleep(wait_time)
```

### 6. Configuration Class

**File:** `app/services/insert_menu_items/config.py`

```python
from dataclasses import dataclass
from typing import Dict, Any
from ...shared.config.loader import get_env_var


@dataclass
class InsertMenuItemsServiceConfig:
    """Configuration for insert_menu_items service"""

    enabled: bool
    bearer_token: str
    create_menu_url: str
    timeout: float
    max_retries: int
    temp_directory: str
    cleanup_after_seconds: int

    @classmethod
    def from_config(cls, config: Dict[str, Any]) -> "InsertMenuItemsServiceConfig":
        """Create config from loaded YAML configuration"""
        service_config = config["services"].get("insert_menu_items", {})

        if not service_config.get("enabled", False):
            raise ValueError("insert_menu_items service is not enabled")

        auth_config = service_config.get("auth", {})
        external_api_config = service_config.get("external_api", {})
        storage_config = service_config.get("storage", {})

        return cls(
            enabled=service_config["enabled"],
            bearer_token=get_env_var(auth_config.get("bearer_token_env", "INSERT_MENU_ITEMS_SERVICE_KEY")),
            create_menu_url=external_api_config.get(
                "create_menu_url",
                "https://zavedenia.com/apps/orderapprestaurant/api/createMenu.php"
            ),
            timeout=external_api_config.get("timeout", 30.0),
            max_retries=external_api_config.get("max_retries", 3),
            temp_directory=storage_config.get("temp_directory", "/tmp/menu_images"),
            cleanup_after_seconds=storage_config.get("cleanup_after_seconds", 3600)
        )
```

### 7. Routes

**File:** `app/services/insert_menu_items/routes.py`

```python
import uuid
from typing import Annotated
from fastapi import APIRouter, File, Form, UploadFile, Depends, HTTPException, BackgroundTasks
import structlog

from .models.requests import InsertMenuItemsRequest
from .models.responses import InsertMenuItemsResponse
from ...shared.middleware.auth import verify_insert_menu_items_auth
from ...shared.exceptions.base import ValidationException

router = APIRouter(prefix="/menu-items", tags=["insert_menu_items"])
logger = structlog.get_logger("insert_menu_items.routes")

# Service instance will be set by main.py
insert_menu_items_service = None


def set_insert_menu_items_service(service):
    """Set the insert_menu_items service instance"""
    global insert_menu_items_service
    insert_menu_items_service = service


@router.post(
    "/insert",
    response_model=InsertMenuItemsResponse,
    status_code=202,  # Accepted
    summary="Upload menu image and trigger async menu creation",
    dependencies=[Depends(verify_insert_menu_items_auth)]
)
async def insert_menu_items(
    background_tasks: BackgroundTasks,
    file: Annotated[UploadFile, File(..., description="Menu image file")],
    city: Annotated[str, Form(..., description="City identifier (e.g., 'sofia')")],
    token: Annotated[str, Form(..., description="API token for createMenu.php")],
    source_language: Annotated[str, Form()] = "bg",
    target_language: Annotated[str, Form()] = "en",
) -> InsertMenuItemsResponse:
    """
    Upload menu image and trigger asynchronous menu extraction and creation.

    Returns immediately with 202 Accepted status.
    Menu processing happens in the background.

    Flow:
    1. Upload image → Immediate response (202)
    2. Background: Extract menu → Transform data → Call createMenu.php
    """

    # Generate unique request ID for tracking
    request_id = str(uuid.uuid4())

    logger.info(
        "insert_menu_items_request",
        request_id=request_id,
        filename=file.filename,
        city=city,
        source_language=source_language,
        target_language=target_language
    )

    try:
        # Validate file
        if not file.content_type or not file.content_type.startswith("image/"):
            raise ValidationException("Only image files are allowed")

        # Read file content
        image_content = await file.read()

        if len(image_content) == 0:
            raise ValidationException("Uploaded file is empty")

        # Add background task
        background_tasks.add_task(
            insert_menu_items_service.process_menu_upload,
            image_content=image_content,
            filename=file.filename,
            mime_type=file.content_type,
            city=city,
            token=token,
            source_language=source_language,
            target_language=target_language,
            request_id=request_id
        )

        logger.info(
            "insert_menu_items_accepted",
            request_id=request_id,
            city=city
        )

        return InsertMenuItemsResponse(
            status="processing",
            message="Image uploaded successfully. Menu extraction and creation in progress.",
            request_id=request_id
        )

    except ValidationException as e:
        logger.error(
            "insert_menu_items_validation_error",
            request_id=request_id,
            error=str(e)
        )
        raise HTTPException(status_code=400, detail=str(e.message))

    except Exception as e:
        logger.error(
            "insert_menu_items_error",
            request_id=request_id,
            error=str(e)
        )
        raise HTTPException(status_code=500, detail=f"Failed to process request: {str(e)}")
```

### 8. Authentication Middleware Update

**File:** `app/shared/middleware/auth.py` (add new verification function)

```python
async def verify_insert_menu_items_auth(
    authorization: Optional[str] = Header(None)
) -> None:
    """Verify authentication for insert_menu_items service"""
    await _verify_auth(authorization, "insert_menu_items")
```

### 9. Integration into Main App

**File:** `app/main.py` (add imports and service initialization)

```python
# Add to imports section
from .services.insert_menu_items.config import InsertMenuItemsServiceConfig
from .services.insert_menu_items.service import InsertMenuItemsService
from .services.insert_menu_items import routes as insert_menu_items_routes

# Add to service initialization (after menu_extraction_service)
try:
    # Insert Menu Items Service
    insert_menu_items_config = InsertMenuItemsServiceConfig.from_config(config)
    insert_menu_items_service = InsertMenuItemsService(
        config=insert_menu_items_config,
        menu_extraction_service=menu_extraction_service
    )
    insert_menu_items_routes.set_insert_menu_items_service(insert_menu_items_service)
    logger.info("insert_menu_items_service_initialized")
except Exception as e:
    logger.warning("insert_menu_items_service_init_skipped", error=str(e))

# Add to router includes
app.include_router(insert_menu_items_routes.router)

# Update startup event to include new service
@app.on_event("startup")
async def startup_event():
    logger.info(
        "application_startup",
        version="2.0.0",
        services=["translation", "wine_pairing", "menu_extraction", "insert_menu_items"],
        # ... other configs
    )
```

### 10. Environment Variables

Add to `.env` file:

```bash
# Insert Menu Items Service
INSERT_MENU_ITEMS_SERVICE_KEY=your-secret-key-here
```

## Testing

### Manual Testing with curl

```bash
# Test the insert_menu_items endpoint
curl --location 'http://localhost:8000/menu-items/insert' \
--header 'Authorization: Bearer your-secret-key-here' \
--form 'file=@"/path/to/menu-image.jpg"' \
--form 'city="sofia"' \
--form 'token="8800a3a64cfa947c552391f48069cacd"' \
--form 'source_language="bg"' \
--form 'target_language="en"'
```

Expected Response (202 Accepted):
```json
{
  "status": "processing",
  "message": "Image uploaded successfully. Menu extraction and creation in progress.",
  "request_id": "550e8400-e29b-41d4-a716-446655440000"
}
```

### Unit Tests

**File:** `tests/services/test_insert_menu_items_adapter.py`

```python
import pytest
from app.services.insert_menu_items.adapter import CreateMenuAdapter
from app.services.menu_extraction.models.dto import (
    MenuExtractionResponse,
    MenuPayload,
    MenuCategory,
    MenuItem
)


def test_adapter_transform():
    """Test adapter transforms menu extraction response correctly"""

    # Create mock menu extraction response
    menu_response = MenuExtractionResponse(
        menu=MenuPayload(
            restaurant_name="Test Restaurant",
            categories=[
                MenuCategory(
                    name="Salads",
                    description="Fresh salads",
                    items=[
                        MenuItem(
                            name="Caesar Salad",
                            description="Classic Caesar",
                            price="15.00 лв",
                            currency="BGN",
                            serving_size="250g",
                            notes=None
                        )
                    ]
                )
            ]
        ),
        source_language="bg",
        target_language="en",
        model="gpt-4o-mini",
        image_count=1
    )

    # Transform
    result = CreateMenuAdapter.transform(
        menu_response=menu_response,
        city="sofia",
        token="test-token"
    )

    # Assertions
    assert result["city"] == "sofia"
    assert result["token"] == "test-token"
    assert len(result["menu"]) == 1
    assert result["menu"][0]["name"] == "Salads"
    assert len(result["menu"][0]["items"]) == 1
    assert result["menu"][0]["items"][0]["name"] == "Caesar Salad"
    assert result["menu"][0]["items"][0]["items"][0]["cena"] == 15.0
    assert result["menu"][0]["items"][0]["items"][0]["gramaj"] == 250


def test_price_extraction():
    """Test price extraction from various formats"""
    adapter = CreateMenuAdapter()

    assert adapter._extract_price("15.00 лв") == 15.0
    assert adapter._extract_price("€10") == 10.0
    assert adapter._extract_price("20") == 20.0
    assert adapter._extract_price(None) == 0.0
    assert adapter._extract_price("") == 0.0


def test_gramaj_extraction():
    """Test serving size extraction"""
    adapter = CreateMenuAdapter()

    assert adapter._extract_gramaj("250g") == 250
    assert adapter._extract_gramaj("100 гр") == 100
    assert adapter._extract_gramaj("1kg") == 1000
    assert adapter._extract_gramaj(None) == 100
    assert adapter._extract_gramaj("") == 100
```

## Deployment Checklist

1. Add configuration file: `config/services/insert_menu_items.yaml`
2. Set environment variable: `INSERT_MENU_ITEMS_SERVICE_KEY`
3. Create service directory structure under `app/services/`
4. Update `app/main.py` with service initialization
5. Update `app/shared/middleware/auth.py` with new auth function
6. Run tests: `pytest tests/services/test_insert_menu_items_adapter.py`
7. Deploy and monitor logs for background task execution

## Monitoring

Monitor logs for the following events:
- `insert_menu_items_request` - Incoming request
- `insert_menu_items_accepted` - Request accepted (202)
- `menu_upload_processing_started` - Background processing started
- `menu_extraction_completed` - Menu extraction successful
- `menu_data_transformed` - Adapter transformation complete
- `create_menu_api_success` - External API call successful
- `menu_upload_processing_completed` - Full workflow complete

## Error Handling

Errors in background processing are logged but do not affect the immediate response to the client. Monitor logs for:
- `menu_extraction_failed`
- `create_menu_api_failed`
- `menu_upload_processing_failed`

Consider implementing:
1. Dead letter queue for failed requests
2. Retry mechanism with exponential backoff (already implemented)
3. Status endpoint to check processing status by `request_id`
4. Webhook callbacks to notify mobile app of completion

## Future Enhancements

1. **Status Tracking**: Add Redis or database to track request status by `request_id`
2. **Webhooks**: Notify mobile app when processing completes
3. **Batch Processing**: Support multiple images in single request
4. **Image Validation**: Validate image quality before processing
5. **Rate Limiting**: Prevent abuse from mobile apps