diff options
| author | yyamashita <yyamashita@mosquit.one> | 2026-05-06 22:24:38 +0900 |
|---|---|---|
| committer | yyamashita <yyamashita@mosquit.one> | 2026-05-06 22:24:38 +0900 |
| commit | 538fd636e25595d88a958344d285c0e7cf44e530 (patch) | |
| tree | eb2999f355570224fa96877d5043af2ef3ec76ef /app/routes/api.scrape.ts | |
| parent | f817604858891edb79e26459dae884b158774db1 (diff) | |
Async scraping, scrape_logs, and CLI
Background scraping:
- POST /api/scrape returns 202 immediately with run_id; scraping runs async
- GET /api/scrape-status?run_id=xxx polls for results per venue
- scrape_logs table: per-venue status (running/ok/error), events_saved, error, timestamps
CLI (npm run scrape):
- npm run scrape — 全会場をスクレイプ、結果を色付きで出力
- npm run scrape liquid-room — 特定会場のみ
- npm run scrape -- --list — 登録済み会場一覧を表示
- エラー時は exit code 1 + エラーメッセージを dim 表示
Venues page:
- 最終スクレイプ日時・成否をインラインで表示
- 会場ごとの「更新」ボタンを追加
Bug fix: upsertEvent に description/optional fields のデフォルト値を設定し
better-sqlite3 の "Missing named parameter" エラーを解消
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Diffstat (limited to 'app/routes/api.scrape.ts')
| -rw-r--r-- | app/routes/api.scrape.ts | 34 |
1 files changed, 17 insertions, 17 deletions
diff --git a/app/routes/api.scrape.ts b/app/routes/api.scrape.ts index 4071985..f9daa5c 100644 --- a/app/routes/api.scrape.ts +++ b/app/routes/api.scrape.ts @@ -1,37 +1,37 @@ /** - * Resource route: POST /api/scrape - * Triggers scraping for all venues (or a specific one via ?venue_id=xxx). - * Returns JSON results and redirects back if called from a form. + * Resource route: /api/scrape + * + * POST (form action) — バックグラウンドでスクレイプ開始、202 を即時返却 + * GET ?venue_id=xxx — 特定会場のみバックグラウンド開始 + * GET (パラメータなし) — 全会場をバックグラウンド開始 + * + * ステータス確認は /api/scrape-status?run_id=xxx */ import { redirect } from "react-router"; import type { Route } from "./+types/api.scrape"; -import { runAllScrapers, runScraper } from "~/lib/scraper-runner.server"; +import { startAllScrapersAsync, startScraperAsync } from "~/lib/scraper-runner.server"; export async function action({ request }: Route.ActionArgs) { const formData = await request.formData(); const venueId = formData.get("venue_id"); - const results = venueId - ? [await runScraper(String(venueId))] - : await runAllScrapers(); + const run_id = venueId + ? startScraperAsync(String(venueId)) + : startAllScrapersAsync(); - // If called from a browser form, redirect back const referer = request.headers.get("Referer"); - if (referer) { - return redirect(referer); - } + if (referer) return redirect(referer); - return Response.json({ results }); + return Response.json({ run_id, status: "started" }, { status: 202 }); } -// Allow GET for quick testing in the browser export async function loader({ request }: Route.LoaderArgs) { const url = new URL(request.url); const venueId = url.searchParams.get("venue_id"); - const results = venueId - ? [await runScraper(venueId)] - : await runAllScrapers(); + const run_id = venueId + ? startScraperAsync(venueId) + : startAllScrapersAsync(); - return Response.json({ results }); + return Response.json({ run_id, status: "started" }, { status: 202 }); } |
