Wikimedia Developer Support

Using Scribunto from Lua script run on command line

I’m extracting Modules from a Wiktionary XML dump and saving them locally in my filesystem in order to get the data required for expanding templates.

For example to get all the language data, I extracted the pages Module:language, Module:language/data, and the other related Modules by recursively finding all dependencies. I saved them locally in the structure that they appear, so “language” would be the top folder name, “data.lua” would be under that.
Then I made a “main.lua” to run the files from command line and output JSON.

All of this works until one of the modules need the mw functions, like mw.ustring, and it’s my understanding these come from the globally available mediawiki functions from Scribunto.

Is there a way to import the mw functions into my local Lua files so I can run them from command line?

Many of those functions call into PHP code (through the special mw_interface global, often aliased to php), so I don’t think it will be possible to run them from the Lua launcher directly. If you can set up a MediaWiki install, you might be able to put together a Scribunto maintenance script that accepts a Lua file as a command line argument (though I’m not sure how functions like frame:getTitle() would behave in a maintenance context).

Is there another way to achieve the same result? I.e. to return JSON data and/or expand templates?

I have MediaWiki and Scribunto installed locally and have it running on localhost, but I’m parsing the XML dump via Python, so I’m looking for some way to get data from Modules and expand templates (e.g. labels {{inflection of|…}}).

There’s currently an error with importDump.php (https://phabricator.wikimedia.org/T211450) so I’m unable to import the XML dump right now.

Super hacky but example doing 2+2 https://phabricator.wikimedia.org/P10025 (run inside maintenance directory of mediawiki)

Using curl to contact the scribunto console api module (via the api.php endpoint) might be a better approach

If using the api module, how would I run a Lua script from say the Languages module?
I.e. The Languages module script is located at https://en.wiktionary.org/wiki/Module:languages (but locally, not on live Wiktionary). What would I do to call the api.php endpoint to run one of the functions in that script?

I tried using the api.php endpoint and it sort of worked, but I’m unable to require Modules. Since the uploadDump.php isn’t working for 1.34, I can’t just require(“Module:…”). I tried require with a local file, but that fails as well.