Le Wiki de Nicoanto

Outils pour utilisateurs

Outils du site

Piste : • DSE / NETVIBES Cloud Connector
dse:netvibes_cloud_connector

Différences

Ci-dessous, les différences entre deux révisions de la page.

Lien vers cette vue comparative

Les deux révisions précédentesRévision précédente
Prochaine révision		Révision précédente
dse:netvibes_cloud_connector [2025/03/11 13:41] – nicoantodse:netvibes_cloud_connector [2025/09/02 12:26] (Version actuelle) – [Pré-Requis] nicoanto
Ligne 1: Ligne 1:
 ====== DSE / NETVIBES Cloud Connector ====== ====== DSE / NETVIBES Cloud Connector ======
  
-==== Checking initial crawl errors ====+===== 'er-crawler-manager-cli' script Options ===== 
 + 
 +<code> 
 +Usage: com.ds.apollo.er.crawler.manager.cli.ManagerCliMain [options] [command] 
 +      [command options] 
 +  Options: 
 +    --help 
 + 
 +  Commands: 
 +    init      Initialize managed crawl operations to the cloud tenant 
 +      Usage: init 
 + 
 +    destroy      Stop managed crawl operations to the cloud tenant and delete 
 +            the existing index unit 
 +      Usage: destroy 
 + 
 +    recrawl      Launch a managed re-crawl to the cloud tenant 
 +      Usage: recrawl 
 + 
 +    config      List configurations 
 +      Usage: config [options] 
 +        Options: 
 +          --list-crawlable-datasources 
 +            List crawlable data sources configurations 
 +            Default: false 
 +          --list-crawled-datasources 
 +            List crawled data sources configurations 
 +            Default: false 
 + 
 +    crawl      Start an unmanaged crawl to the cloud tenant 
 +      Usage: crawl [options] 
 +        Options: 
 +        * --strategy 
 +            Crawl strategy 
 +          --monitored 
 +            True if the crawl should be monitored (true by default) 
 +            Default: true 
 + 
 +    test-cloud-connection      Test connection to the cloud tenant services 
 +      Usage: test-cloud-connection 
 + </code> 
 + 
 + 
 +===== Checking initial crawl errors =====
  
 Initial crawl job generates a dataset to disk in $CLOUD_CONNECTOR_INSTALL_DIR/$ARCH/er-crawler/storage/datasets/default, containing all successfully crawled document identifiers as well as all errors that happened during that crawl. Objective of this dataset, is to be able to analyze & replay the crawl to recover errors that happened. Initial crawl job generates a dataset to disk in $CLOUD_CONNECTOR_INSTALL_DIR/$ARCH/er-crawler/storage/datasets/default, containing all successfully crawled document identifiers as well as all errors that happened during that crawl. Objective of this dataset, is to be able to analyze & replay the crawl to recover errors that happened.
Ligne 18: Ligne 61:
   * [[https://help.3ds.com/2025x/english/DSDoc/EXADfsUserMap/exadfs-m-indexing-onpremise-data-sb.htm?contextscope=cloud&id=c858c5ede1374cc89e191130ac3b3db2|DS Doc - Indexing 3DSpace On-premises Data on the Cloud]]   * [[https://help.3ds.com/2025x/english/DSDoc/EXADfsUserMap/exadfs-m-indexing-onpremise-data-sb.htm?contextscope=cloud&id=c858c5ede1374cc89e191130ac3b3db2|DS Doc - Indexing 3DSpace On-premises Data on the Cloud]]
   * [[https://gitlab.paris.exalead.com/apollo/er-crawler/-/wikis/Cloud-Supplemental|Wiki R&D - Cloud Supplemental]]   * [[https://gitlab.paris.exalead.com/apollo/er-crawler/-/wikis/Cloud-Supplemental|Wiki R&D - Cloud Supplemental]]
 +  * [[https://gitlab.paris.exalead.com/apollo/er-crawler/-/wikis/Processing-Configuration|Wiki R&D - Cloud Supplemental - Filtering data]]
   * [[https://dsext001-eu1-215dsi0708-3dswym.3dexperience.3ds.com/community/swym:215dsi0708:DSEXT001:community:-UWQVGuiSeuoZ33WWbTLWQ?content=swym:215dsi0708:DSEXT001:wikitree:34OA6fJ5QwOxg9SBzA9l0Q|EDF Road to Cloud - NETVIBES OnPremise Cloud Connector]]   * [[https://dsext001-eu1-215dsi0708-3dswym.3dexperience.3ds.com/community/swym:215dsi0708:DSEXT001:community:-UWQVGuiSeuoZ33WWbTLWQ?content=swym:215dsi0708:DSEXT001:wikitree:34OA6fJ5QwOxg9SBzA9l0Q|EDF Road to Cloud - NETVIBES OnPremise Cloud Connector]]
   * [[https://dsext001-eu1-215dsi0708-3dswym.3dexperience.3ds.com/community/swym:215dsi0708:DSEXT001:community:-UWQVGuiSeuoZ33WWbTLWQ?content=swym:215dsi0708:DSEXT001:wikitree:IHOjbqGPRKKwDT1N52GpeA|EDF Road to Cloud - Administration OP - RAZ & Indexation Full Cloud Connector]]   * [[https://dsext001-eu1-215dsi0708-3dswym.3dexperience.3ds.com/community/swym:215dsi0708:DSEXT001:community:-UWQVGuiSeuoZ33WWbTLWQ?content=swym:215dsi0708:DSEXT001:wikitree:IHOjbqGPRKKwDT1N52GpeA|EDF Road to Cloud - Administration OP - RAZ & Indexation Full Cloud Connector]]
   * [[https://dsext001-eu1-215dsi0708-3dswym.3dexperience.3ds.com/community/swym:215dsi0708:DSEXT001:community:-UWQVGuiSeuoZ33WWbTLWQ?content=swym:215dsi0708:DSEXT001:wikitree:TEVNkZegQTeaF_PMSN9Xjw|EDF Road to Cloud - (Internal) Unboxing DATA SCIENCE R2023X/R2024x]]   * [[https://dsext001-eu1-215dsi0708-3dswym.3dexperience.3ds.com/community/swym:215dsi0708:DSEXT001:community:-UWQVGuiSeuoZ33WWbTLWQ?content=swym:215dsi0708:DSEXT001:wikitree:TEVNkZegQTeaF_PMSN9Xjw|EDF Road to Cloud - (Internal) Unboxing DATA SCIENCE R2023X/R2024x]]
 +
 +
 +===== Installation on vdemopro =====
 +
 +==== Pré-Requis ====
 +
 +En tant que l'utilisateur 'e4apop', dans le répertoire /home/data/RTV/DEMOR424/, il faut :
 +  * créer le répertoire '/home/data/RTV/DEMOR424/netvibesCloudConnector'
 +  * copier le tar du connecteur et le décompresser dans le répertoire '/home/data/RTV/DEMOR424/netvibesCloudConnector'
 +  * y copier le tar de tomee et le décompresser dans le répertoire '/home/data/RTV/DEMOR424/netvibesCloudConnector'
 +==== Exécution du script d'installation ====
 +
 +Il faut lancer le script : ./NETVIBES_CloudConnector.Linux64/1/StartTUI.sh
 +<code>
 +[e4apop@vdemopro083abv netvibesCloudConnector]$ ./NETVIBES_CloudConnector.Linux64/1/StartTUI.sh
 +CurrentMediaDir initial="./NETVIBES_CloudConnector.Linux64/1"
 +CurrentMediaDir="/home/data/RTV/DEMOR424/netvibesCloudConnector/./NETVIBES_CloudConnector.Linux64/1"
 +Current operating system: "Linux"
 +/usr/bin/lsb_release
 +DSY_OS_Release="RedHatEnterprise"
 +DSY_OS="linux_a64"
 +DSY_LIBPATH_VARNAME="LD_LIBRARY_PATH"
 +Check Prerequisites for Linux
 +RedHatEnterprise 8.4
 +Check Prerequisites for Linux OK.
 + INFO: /home/data/RTV/DEMOR424/netvibesCloudConnector/./NETVIBES_CloudConnector.Linux64/1/inst/linux_a64/code/bin/DSYInsAppliTUI
 + INFO: Installer found in current media: "/home/data/RTV/DEMOR424/netvibesCloudConnector/./NETVIBES_CloudConnector.Linux64/1/inst/linux_a64/code/bin/DSYInsAppliTUI".
 +
 +
 +
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Welcome to the 3DEXPERIENCE R2022x HotFix 6 installation program.
 +
 +Please refer to the documentation for the Legal Notices.
 +Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Enter selection (default: Next):
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Choose the installation directory to update or an empty directory to do a new installation:
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [/usr/DassaultSystemes/R2022x/NETVIBES_CloudConnector]: /home/data/RTV/DEMOR424/netvibesCloudConnector/NETVIBES_CloudConnector
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Apache TomEE+ installation path
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default []: /home/data/RTV/DEMOR424/netvibesCloudConnector/apache-tomee-plus-8.0.15
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +The embedded Apache TomEE+ will be installed for Testing or Demonstration purposes (not for Production).
 +
 +Apache TomEE+ connection port
 +Input !b to get the previous page. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [8080]: 8080
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Choose the server installation directory.
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [/usr/DassaultSystemes/R2022x/3DSpace]: /home/data/RTV/DEMOR424/server
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the 64-bit Java Development Kit (JDK) 11 path.
 +The minimum required Java version is Oracle JDK 11.0.8 / AdoptOpenJDK-OpenJ9 JDK 11.0.8 (or its IBM Semeru successor) / Oracle JDK 17.0.3 / IBM Semeru JDK 17.0.3, but it is recommended to install the latest Update of Java 11.
 +
 +To download and install the JDK 11, go to https://www.oracle.com/java/technologies/downloads/#java11 or https://developer.ibm.com/languages/java/semeru-runtimes/downloads/.
 +It is strongly recommended to use a Java installation path independent from the Java level.
 +
 +After the Java installation completes, select '< Back' then 'Next >' again to auto-detect the Java path.
 +
 +Input !b to get the previous page. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default []: /home/data/RTV/DEMOR424/jdk
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the username of the crawl user
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [creator]: creator
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the password for the crawl user
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [****]: DemoCenter01
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Do you want to require users to have a valid CAS session ?
 +Input !b to get the previous page. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +1 [X] No
 +Enter selection:
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the URL of the on cloud 3DRegistry
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [https://GEO-registry.3dx-staging.3ds.com]: https://eu1-registry.3dexperience.3ds.com
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the name of the cloud tenant
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default []: r1132101374770
 +
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the on cloud Workspace ID
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [dw-prj-111111-ws]: ffd38927-6958-4bc5-bc85-b7799e71c2dc
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the Agent ID of an authorized CLM agent to call SGI on the cloud
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default []: fa9f240d-9f14-400a-9d33-da5bf053936b
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +Enter the password for CLM agent
 +Input !b to get the previous page. Input !c to clear the default value. Input !q and hit Enter to quit. Just press Enter to continue.
 +
 +Default [****]: B<n(QU+S+z)"cbM"X[%v3CN}
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +The installation program has enough information to start.
 +Input !b to get the previous page. Input !q and hit Enter to quit. Just press Enter to start installation.
 +
 +Files will be installed in the following directory:
 +  /home/data/RTV/DEMOR424/netvibesCloudConnector/NETVIBES_CloudConnector
 +Apache TomEE+ installation path: /home/data/RTV/DEMOR424/netvibesCloudConnector/apache-tomee-plus-8.0.15
 +The embedded Apache TomEE+ will be installed for Testing or Demonstration purposes (not for Production).
 +  Apache TomEE+ connection port: 8080
 +The server installation directory: /home/data/RTV/DEMOR424/server
 +Java Development Kit (JDK) path: /home/data/RTV/DEMOR424/jdk
 +Crawl username: 'creator'
 +CAS session validation: 'no'
 +Cloud Registry URL: 'https://eu1-registry.3dexperience.3ds.com'
 +Cloud Tenant Name: 'r1132101374770'
 +Cloud Workspace ID: 'ffd38927-6958-4bc5-bc85-b7799e71c2dc'
 +Cloud CLM Agent ID: 'fa9f240d-9f14-400a-9d33-da5bf053936b'
 +
 +Space available: 64.56 GB
 +Space required: 41.18 MB
 +
 +Enter selection (default: Install):
 +Logs are located in "/home/data/RTV/DEMOR424/netvibesCloudConnector/NETVIBES_CloudConnector/InstallData/log/20250902T103230+0200_CODE_linux_a64_NETVIBES_CloudConnector.media"
 +Copying files...
 +Installation in progress...
 +Installing files...
 +CODE\linux_a64\EXAApoERCrawlerInsCrawlPwCfg (161B)
 +CODE\linux_a64\EXAApoERCrawlerPreq (63B)
 +CODE\linux_a64\EXAApoERCrawlerCheckers (5KB)
 +CODE\linux_a64\EXAApoERCrawlerCommons (103KB)
 +CODE\linux_a64\EXAApoERCrawlerService (1MB)
 +CODE\linux_a64\VENSLF4J (787KB)
 +CODE\linux_a64\RegistryClient (41KB)
 +CODE\linux_a64\iamCASClient (215KB)
 +CODE\linux_a64\ENOJavaKernel (2MB)
 +CODE\linux_a64\ENOFCSJavaClient (441KB)
 +CODE\linux_a64\VENFasterxmlJackson (4MB)
 +CODE\linux_a64\VENGoogleGuava (2MB)
 +CODE\linux_a64\VENApacheCommonsIO (489KB)
 +CODE\linux_a64\VENApacheCommonsCodec (327KB)
 +CODE\linux_a64\VENApacheCommonsHttpClient (1MB)
 +CODE\linux_a64\VENGoogleProtoBuf (1MB)
 +CODE\linux_a64\VENGson (252KB)
 +CODE\linux_a64\VENYammerMetrics (83KB)
 +CODE\linux_a64\EXASupplementalPreq (63B)
 +CODE\linux_a64\EXASupplementalCommons (3MB)
 +CODE\linux_a64\EXAApoERCrawlerPreq4OP (14MB)
 +CODE\linux_a64\EXAApoERCrawlerDataSetCLI (44KB)
 +CODE\linux_a64\EXAApoERCrawlerManager (183KB)
 +CODE\linux_a64\EXAApoERCrawlerManagerCLI (76KB)
 +CODE\linux_a64\EXAApoERCrawlerFileSyncSvc (92KB)
 +CODE\linux_a64\EXAApoERCrawlerManagerWebapp (6KB)
 +CODE\linux_a64\EXAApoERCrawlerServiceWebapp (11KB)
 +CODE\linux_a64\EXAApoERCrawlerInstaller (88KB)
 +CODE\linux_a64\ERConnectorCommon (239KB)
 +CODE\linux_a64\ERAgent (1MB)
 +CODE\linux_a64\VENJavaEE_API (4MB)
 +CODE\linux_a64\VENApacheJohnzon (253KB)
 +CODE\linux_a64\VENApacheCommonsLang (688KB)
 +CODE\linux_a64\NETECOP_TP.prd (6KB)
 +Installation in progress...
 +Setting up properties file...
 +Customizing file...
 +Executing application...
 +Generating er-crawler-manager-cli classpath...
 +Generating er-crawler-dataset-cli classpath...
 +Generating er-crawler-service-webapp.war...
 +Generating er-crawler-manager-webapp.war...
 +Customizing file...
 +Executing application...
 +Installing er-crawler...
 +
 +______________________________________
 +
 +   Success
 +______________________________________
 +
 +Installed er-crawler, you can now start your application server
 +Once your application server is up, you can initialize the crawl of your tenant by running the following command:
 +        '/home/data/RTV/DEMOR424/netvibesCloudConnector/NETVIBES_CloudConnector/linux_a64/code/command/er-crawler-manager-cli.sh init'
 +
 +Actions:
 + [1] Ok
 +______________________________________
 +
 +Please choose an action: 1
 +Setting up properties file...
 +Installation in progress...
 +
 +
 +
 +
 +
 +________________________________________________________________________________
 +
 +   3DEXPERIENCE R2022x HotFix 6
 +________________________________________________________________________________
 +
 +The installation program has finished installing 3DEXPERIENCE R2022x HotFix 6 on your computer, in /home/data/RTV/DEMOR424/netvibesCloudConnector/NETVIBES_CloudConnector.
 +
 +Just press Enter to exit installation.
 +
 +Installed Components:
 +  NETVIBES ER Crawler On Premise TP
 +
 +
 +Enter selection (default: Close):
 +
 +</code>
 +
 +==== Modifications de la config Apache ====
 +En tant que 'root', aller dans le répertoire '/usr/local/reverseproxy/' et modifier le fichier 'conf/httpd.conf
 +' en ajoutant, avant le tag </VirtualHost> les lignes suivantes :
 +<code>
 +  # Analytics Applications
 +  ProxyPass        /connector http://vdemoproXXXdsy.extranet.3ds.com:8080
 +  ProxyPassReverse /connector http://vdemoproXXXdsy.extranet.3ds.com:8080
 +
 +</code>
 +
 +Redémarrer le proxy avec une des commandes suivantes
 +<code>
 +bin/apachectl restart
 +</code>
 +ou
 +<code>
 +/usr/local/reverseproxy/bin/httpd -k restart
 +</code>
 +
  
dse/netvibes_cloud_connector.1741700477.txt.gz · Dernière modification : (modification externe)
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki