Hi Robert, welcome to the list. Opening big files is definitely tricky. I
believe (not sure) folks on this list usually write scripts to filter and
aggregate these files, or load them into big data tools. But I found
someone else asking a similar question with a very useful answer here:
https://stackoverflow.com/questions/159521/text-editor-to-open-big-giant-hu…
On Thu, Jan 26, 2023 at 11:04 AM Robert Garrigos <robert(a)garrigos.cat>
wrote:
Hi,
I just enrolled this list, thanks to Dan Andreescu, who let me know
about it, and I have a question on processing clickstream data.
I downloaded a file for last month clickstream data
(
https://dumps.wikimedia.org/other/clickstream/2022-12/clickstream-eswiki-20…)
and have problems to open it and processing it.
The only programme I could open it was OpenRefine. Other programmes
(Numbers and LibreOffice) just couldn't cope with it.
I can use OpenRefine to do some transformation and delete some rows I
don't need, but even then, with some 1.5milion rows, I can not open it
with numbers or libreoffice to do sum of the column 4.
Which tools do you use to work with such big files?
Thanks.
--
========================
Robert Garrigós i Castro
https://garrigos.cat
+34 620 91 87 01
_______________________________________________
Analytics mailing list -- analytics(a)lists.wikimedia.org
To unsubscribe send an email to analytics-leave(a)lists.wikimedia.org