Formulasearch

A Formulasearch BaseX REST endpoint for enabling the formulasearch extension in mediawiki. This is initially based on: https://github.com/ag-gipp/vmext-demo/tree/fix_endpoints_for_formulasearch

Using env file locally

With IntelliJ Idea, download and install env file plugin: https://plugins.jetbrains.com/plugin/7861-envfile/ rename template.env to local.env, change to local params and set it in application start in IntelliJ.

local development

Create a directory in /opt/basex and grant correct access permissions for storing logs and prevent warnings.

Note for Checkstyle

In IntelliJ IDEA: Install Checkstyle IDEA Plugin, In checkstyle settings load the checkstyle.xml-file, set checkstyle version =~8.29 to ensure compatibility.

================================================================

MathML Pipeline

This project is a Java application (Spring Boot) with an embedded Apache Server. It takes on the following tasks:

Convert LaTeX formula to MathML semantics (via MathMLConverters).
Compare two MathML semantics and receive node similarities (via tree comparison on MathML).
Render a single MathML oder the comparison results of both MathML (via a JS widget).

Task 1 and 2 are executed via an embedded REST Service in this project and is written in Java. Task 3 is executed via an external JS widget written by students at the HTW Berlin.

Build

Initialize the submodules

git submodule update --init --recursive

Do maven build

Check-out this project and build it via Maven. In the /target directory you will find the executable server instance mathpipeline.jar.

mvn clean install

Maven 3.2 or higher is required for the build plugin, it will create an executable jar.

Requirements

For the conversion from LaTeX to MathML, LaTeXML is required. Via configuration you can choose to use an external service for the LaTeX > MathML conversion or use a local installation of LaTeXML.

You can find LaTeXML here: http://dlmf.nist.gov/LaTeXML/get.html.

Installation (Standalone)

Copy the Jar from the target folder to wherever you want.

Start the server by java -jar mathpipeline.jar.

Now just call the server on http://localhost:8080/index.html and you can start whatever you what to do.

Installation (Service)

Since this is a Spring Boot application it can easily be used as a Service in a Linux environment, see: https://docs.spring.io/spring-boot/docs/current/reference/html/deployment-install.html

Copy the Jar from the target folder to /var/mathpipeline/

Simply create a symlink (change the path towards your installation)

$ sudo ln -s /var/mathpipeline/mathpipeline.jar /etc/init.d/mathpipeline
Once installed, you can start and stop the service in the usual way, e.g. service mathpipeline [start|stop|status|restart]

You can find an automatic log in /var/log/mathpipeline.log.

JVM Options

It is recommended to limit your JVM and set VM options. Just place a file called mathpipeline.conf besides the mathpipeline.jar and add the following content:

export JAVA_OPTS="-Xmx512m -Xms256m"

It should then look like this:

$ <your_installation_path>    
$ ls
mathpipeline.jar
mathpipeline.conf
$ cat mathpipeline.conf
export JAVA_OPTS="-Xmx512m -Xms256m"

REST API

We use Swagger for the API documentation. You can always view the API per default on http://localhost:8080/swagger-ui.html.

Configuration

If you want to use a custom configuration place a file named application.yaml in the execution / installation folder. The content should be (this is the default configuration used by the application):

server:
#  servlet-path: /pipe   # custom servlet-path
  port: 8080            # default server port, if not set otherwise

# Math AST Renderer - Main URL
mast.url: http://math.citeplag.org

# the use of the latexml online service (via url)
# if no url is given a local latexml installation will be called
latexml:
  active: true
  url: http://gw125.iu.xsede.org:8888 # url for online service or emtpy ""
  params:                             # parameters for the online service
    whatsin: math
    whatsout: math
    includestyles:
    format: xhtml
    pmml:
    cmml:
    nodefaultresources:
    linelength: 90
    quiet:
    preload:
      - "LaTeX.pool"
      - "article.cls"
      - "amsmath.sty"
      - "amsthm.sty"
      - "amstext.sty"
      - "amssymb.sty"
      - "eucal.sty"
      - "DLMFmath.sty"
      - "[dvipsnames]xcolor.sty"
      - "url.sty"
      - "hyperref.sty"
      - "[ids]latexml.sty"
      - "texvc"

# Mathoid - alternative Latex to MathML converter
mathoid:
  active: true
  url: http://localhost:10044/mml

Deploy for DKE

The vmext-demo docker is now a private docker container. It is no longer maintained over DockerHub because it contains LaCASt (currently private).

Updating vmext-demo on DKE has to be done manually now.

checkout the repo and build it so that you have a running target/mathpipeline.jar
Check if its working locally. You need to update lacast.config.yaml according to your system needs. You may need elasticsearch and mathoid running locally also or you create port-forwarding and connect to DKE via ssh.
Run locally java -jar target/mathpipeline.jar and check on localhost:8080/swagger-ui.html if everything works as expected.
Bring the changed files to DKE via scp target/mathpipeline user@dke01:~/mathpipeline.jar
Login to dke server via ssh
Find the running docker container via docker ps. The name is vmext-demo and tag lacast.
Copy the changed files to the appropriate locations inside the docker container via docker cp mathpipeline.jar vmext-demo:/mathpipeline.jar
IMPORTANT: update the image of the container after you changed something in the container via docker commit vmext-demo vmext-demo:lacast (first the docker name, than the image name)
Restart the container so that the changes gets applied via /usr/bin/docker-compose restart vmext-demo.

That's it.

If you need to start from scratch you need a free developer license for mathematica and: 1. Again, build everything locally and test if it works. 2. Build the container docker build . -t vmext-demo:lacast (thats the image name). This may take some time because installing mathematicas license requires some basic ubuntu libraries. 3. Run the docker container for the first time docker run --name vmext-demo -p 8080:8080 vmext-demo:lacast 4. Now connect to the running container docker exec -it vmext-demo /bin/bash 5. Run /usr/bin/wolframscript and activate your license (e-mail credentials you took to get your free developer license. The credentials are the same you need on the website. The wolfram ID is you e-mail.) 6. exit with exit 7. Very important: update your container NOW! otherwise on the next restart its gone... docker commit vmext-demo vmext-demo:lacast. 8. That's it. For deployment and updates, you need to follow the guide above.

GitHub:Formulasearch