Skip to content

README.md

Latest commit

 

History

History
183 lines (157 loc) · 6.34 KB

README.md

File metadata and controls

183 lines (157 loc) · 6.34 KB

smm-extractor

This repo includes a few popular Social Media Macroscope Extractors such as Sentiment Analysis, Network Visualization, Text Preprocessing, Topic Modeling, Name Entity Recognition, and etc.

Build

To build Clowder SMM extractors, make sure to use the correct dockerfile in each of the extractor folder (extractor. dockerfile).

  • Note you can also build SMM extractor (dockerfile) that is being used independently of Clowder but on the SMM system.

Some of the useful command:

Build

docker build -f extractor.dockerfile -t socialmediamacroscope/network_analysis_extractor:latest .

Push

docker push socialmediamacroscope/network_analysis_extractor:latest

Deploy

For how to deploy Clowder extractors please refer to the official [Documentation](https://clowder-framework. readthedocs.io/en/latest/develop/extractors.html#.)

You can access the latest pre-built SMM extractors on docker.io:

socialmediamacroscope/sentiment_analysis_extractor socialmediamacroscope/topic_modeling_extractor socialmediamacroscope/network_analysis_extractor socialmediamacroscope/preprocessing_extractor socialmediamacroscope/name_entity_recognition_extractor

To include those extractors with docker-compose:

  smm-sentiment:
    image: socialmediamacroscope/sentiment_analysis_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  smm-preprocessing:
    image: socialmediamacroscope/preprocessing_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  name-entity-recognition:
    image: socialmediamacroscope/name_entity_recognition_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  network-analysis:
    image: socialmediamacroscope/network_analysis_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  topic-modeling:
    image: socialmediamacroscope/topic_modeling_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

Submit file to extractors

smm-extractor

This repo includes a few popular Social Media Macroscope Extractors such as Sentiment Analysis, Network Visualization, Text Preprocessing, Topic Modeling, Name Entity Recognition, and etc.

Build

To build Clowder SMM extractors, make sure to use the correct dockerfile in each of the extractor folder (extractor. dockerfile).

  • Note you can also build SMM extractor (dockerfile) that is being used independently of Clowder but on the SMM system.

Some of the useful command:

Build

docker build -f extractor.dockerfile -t socialmediamacroscope/network_analysis_extractor:latest .

Push

docker push socialmediamacroscope/network_analysis_extractor:latest

Deploy

For how to deploy Clowder extractors please refer to the official [Documentation](https://clowder-framework. readthedocs.io/en/latest/develop/extractors.html#.)

You can access the latest pre-built SMM extractors on docker.io:

socialmediamacroscope/sentiment_analysis_extractor socialmediamacroscope/topic_modeling_extractor socialmediamacroscope/network_analysis_extractor socialmediamacroscope/preprocessing_extractor socialmediamacroscope/name_entity_recognition_extractor

To include those extractors with docker-compose:

  smm-sentiment:
    image: socialmediamacroscope/sentiment_analysis_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  smm-preprocessing:
    image: socialmediamacroscope/preprocessing_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  name-entity-recognition:
    image: socialmediamacroscope/name_entity_recognition_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  network-analysis:
    image: socialmediamacroscope/network_analysis_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}

  topic-modeling:
    image: socialmediamacroscope/topic_modeling_extractor:latest
    restart: unless-stopped
    networks:
      - clowder
    depends_on:
      - rabbitmq
      - clowder
    environment:
      - RABBITMQ_URI=${RABBITMQ_URI:-amqp://guest:guest@rabbitmq/%2F}