Say I have a large txt or CSV file with data I want to search. And say I have several files.

What is the best way to index and make this data searchable? I’ve been using grep, but it is not ideal.

Is there any self hostable docker container for indexing and searching this? Or maybe should I use SQL?

Eager Eagle
link
fedilink
English
3
edit-2
4M

Excel / OnlyOffice?

I love self-hosted tools, but you can do a lot on a spreadsheet.

Btw, if the files are not too large, you can query them using SQL without even hosting a database just by using Pandas. This avoids the problem of updating entries and handling migrations in case the CSVs change over time.

@Dust0741@lemmy.world
creator
link
fedilink
14M

Files won’t change and are hundreds of GBs

Are they roughly 55GB compressed?

Spill the beans!

@Dust0741@lemmy.world
creator
link
fedilink
24M

👀

Eager Eagle
link
fedilink
English
14M

ok, database it is then

Self Hosted - Self-hosting your services.
!selfhost@lemmy.ml
Create a post

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.

Rules

  • No harassment
  • crossposts from c/Open Source & c/docker & related may be allowed, depending on context
  • Video Promoting is allowed if is within the topic.
  • No spamming.
  • Stay friendly.
  • Follow the lemmy.ml instance rules.
  • Tag your post. (Read under)

Important

Beginning of January 1st 2024 this rule WILL be enforced. Posts that are not tagged will be warned and if not fixed within 24h then removed!

  • Lemmy doesn’t have tags yet, so mark it with [Question], [Help], [Project], [Other], [Promoting] or other you may think is appropriate.

Cross-posting

If you see a rule-breaker please DM the mods!

  • 0 users online
  • 2 users / day
  • 2 users / week
  • 5 users / month
  • 38 users / 6 months
  • 1 subscriber
  • 168 Posts
  • 303 Comments
  • Modlog