ATG - Bcc Full Deployment Failure with IOException

Error

Deployments Fail Due to Corruption in Version File Store, IOException in Log: Unexpected end of stream while reading deployment protocol

Applies to:

Oracle Commerce Platform - Version 9.4 and later
Information in this document applies to any platform.

Symptoms

Deployments fail as a result of problems with a file asset, such as targeter or scenario .properties file, in the VersionFileStore .  In the BCC UI, the deployment status shows that the deployment failed and references a versioned file that the VersionManagerService is trying to reference.

Deployment Failed

Deployment Status: Error Applying
Error Message:
Deployment FailedDeployment 'ID' to target 'TARGET' encountered a system level deployment error during data transfer.

Asset:
uri:atgvm://VersionManagerService/PublishingFiles/targeter/fa108888 (note URI maybe different for your project)

Note: The error "atg.deployment.common.DeploymentException: Deployment 'ID' to target 'TARGET' encountered a system level deployment error during data transfer" is a generic error for any deployment failure and is not specific to only this issue.

Cause

It is possible, but not mandatory, that a problem occurred during a previous deployment. For example, a deployment was run and a target was restarted in the middle, or a previous deployment failed. It is also possible that no specific deployment problem was detected.

Solution

Note:  Exercise caution when directly modifying the versioned database schema.  For assistance identifying this problem and implementing the solution, please open an SR with Oracle Support.

1. Follow the steps in sequence

• login into PUB schema with sql developer or ssh on particular environment.
• take backup/export of table epub_file_asset table content
• run following command <update epub_file_asset set VERSION_DELETE=1;
• commit the changes
• invalidate the caches on PublishingRepository through Dyn/Admin of BCC instance.
• Error will disappear

Issues During Code Build in Eclipse

Problem

If you are facing issue during Build a project in Eclipse and you cannot see anything in Problems perspective of eclipse after you checkout project from SVN Repository.

Solution

Please follow the steps as below to successfully checkout the project and build in eclipse.

Note : We are not referring to ANT Build in eclipse.

Steps :

1. Checkout the project from SVN repository. By giving correct SVN path.
2. On the Select Wizard Screen on eclipse, select as Java Project.
3. Give Name to the project and select current used JRE and click finish.
4. After project is successfully check out, right click on the project and select "Properties".
5. Select Java BUILD Path from Left Menu and click add Folders.
6. Select all "Src" folders from all modules present in your Project.
7. Click "OK".
8. Instead of "Build Automatically" "Clean" Manual Build is Recommended.
9. Now you can see problems in your perspective.
10. Go Ahead and add specific libraries required for compilation.

Error Starting Dyn Admin on Prod. ( java.io.IOException: Cannot run program "javac")

Error

Error : The detailed exception is: java.io.IOException: Cannot run program "javac": CreateProcess error=206, The filename or extension is too long

atg.servlet.pagecompile.jsp11.JSPPageProcessor.javaToClass(JSPPageProcessor.java:421) [_DAS_slib_sclasses.jar:]

APPLIES TO:

Oracle Commerce Platform - Version 9.4 and later
Information in this document applies to any platform.

CAUSE

Above error is Java compiler error, where java compiler fails to compile.

SOLUTION

1. Follow the steps in sequence

• Go to location "servers/atg_config/local_prod1a_cbsbuild/localconfig/atg/dynamo/servlet" in your code base.
• Add folder pagecompile under servlet.
• Add two files with name "ExtendedJhtmlPageProcessor.properties" and "PageProcessor.properties"
• In Extend file add following code snippet
  $class = atg.servlet.pagecompile.jsp11.JSPPageProcessor
  javaCompilerClassName=atg.servlet.pagecompile.SunJavaSourceCompiler 
  
• In PageProcessor.properties add following code
  javaCompilerClassName=atg.servlet.pagecompile.SunJavaSourceCompiler
• Build your project again and restart the server.

java.lang.NoClassDefFoundError: com/sun/tools/javac/Main

error

JBWEB000309: type JBWEB000066: Exception report

JBWEB000068: message JBWEB000248: Servlet execution threw an exception

JBWEB000069: description JBWEB000145: The server encountered an internal error that prevented it from fulfilling this request.

JBWEB000070: exception

javax.servlet.ServletException: JBWEB000248: Servlet execution threw an exception

JBWEB000071: root cause

java.lang.NoClassDefFoundError: com/sun/tools/javac/Main
atg.servlet.pagecompile.SunJavaSourceCompiler.javaToClass(SunJavaSourceCompiler.java:110)
atg.servlet.pagecompile.jsp11.JSPPageProcessor.javaToClass(JSPPageProcessor.java:405)
atg.servlet.pagecompile.PageProcessor.compilePageClass(PageProcessor.java:2437)
atg.servlet.pagecompile.PageProcessor.getPageInfo(PageProcessor.java:1787)
atg.servlet.pagecompile.jsp12.Jsp12PageProcessor.getPageInfo(Jsp12PageProcessor.java:324)
atg.servlet.minimal.MinimalServletContainer.compileServlet(MinimalServletContainer.java:153)
atg.servlet.pagecompile.PageCompileServlet.service(PageCompileServlet.java:306)

APPLIES TO

Oracle Commerce Platform - Version 9.4 and later
Information in this document applies to any platform.

SOLUTION

1. Follow the steps in sequence

• create folder structure under your jboss/modules as "com\sun\tools\javac\main"
• Copy and paste attached one file to this location. "module.xml" & Copy tools.jar from jre/lib location on your system.module.xml
• Copy "tools.jar" to jboss_home/standalone/deployments/<prod_server>.
• Copy "tools.jar" to code base location "/<module>/<sub-module>/lib"
• Go to code base location "<project>/<module>/META-INF" and open file "MANIFEST.mf"
• Add "lib/tools.jar" to ATG-Class-Path in the file.
• Stop Servers if running & Re-Build the ear's.
• Restart the server and make sure <module>.<sub-module> is started with parameters in Server Startup.
• Similar way can fix Pub if there is same error.
• create folder structure under your jboss/modules as "com\sun\tools\javac\main".

Could not load definitions from resource net/sf/antcontrib/antcontrib.properties

error

When click on build.xml and running ant build. Getting following error :

[taskdef] Could not load definitions from resource net/sf/antcontrib/antcontrib.properties. It could not be found.

APPLIES TO:

Oracle Commerce Platform - Version 9.4 and later
Information in this document applies to any platform.

Solution

Please follow the steps below to get rid of above issue.

Important steps to be followed and are mandatory.

1. Install ANT on to your machine drive.
2. Make sure class path for ant is set to ANT_HOME
3. In lib folder of ANT_HOME copy jar file names "ant-contrib-0.6".
4. In build.xml at line number <exact line number> where it say's "
   
   <taskdef resource="net/sf/antcontrib/antcontrib.properties">
   <classpath>
   <pathelement location="" /> <taskdef>
5. Give correct location of jar files. e.g "C:\apache-ant-1.8.4\lib\ant-contrib-0.6.jar".
6. Try running ANT build again.

java.io.StreamCorruptedException: unexpected block data

Error

We were getting below error when we try to start the ACC in our local VM

java.io.StreamCorruptedException: unexpected block data
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1364)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at atg.core.util.UnsynchronizedHashtable.readObject(UnsynchronizedHashtable.java:623)
at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream

APPLIES TO:

Oracle Commerce Platform - Version 9.4 and later
Information in this document applies to any platform.

Cause

Some of the SQLrepositories were modified with custom properties descriptors and ACC is not able to find them while start up.

SOLUTION

Please configure the application module with the following entries in the module's MANIFEST.MF file:

ATG-Client-Class-Path: A listing of the jar files that will added to the ACC client's classpath
Name: The filename of each jar file entry in the ATG-Client-Class-Path that needs to be downloaded
ATG-Client-Update-File: A true or false flag whether or not the filename should be downloaded by the ACC
ATG-Client-Update-Version: Optional version string used to force the ACC to update the specified filename

If you see the MANIFEST.MF file in PCIencryption module and it doesn't have the above entry. So please add the below 3 entries in the MANIFEST.MF file of PCIencryption Module.

ATG-Client-Class-Path: bin/
Name: bin/
ATG-Client-Update-File: true

ATG-Version is not required.  Do a build and start the server and try to access ACC through script or dyn/admin . You should be able to access it.

SEVERE: Caught an exception while invoking method 'run' on object 'BaselineUpdate'

Error:

Getting following error during ./basline_update.sh

Error : 

[[

8| Dgidx.cleanDirs();
9|
10| // run crawl and archive any changes in dvalId mappings
11| CAS.runBaselineCasCrawl("CBS-last-mile-crawl");
12| CAS.archiveDvalIdMappingsForCrawlIfChanged("CBS-last-mile-crawl");
13|
14| // archive logs and run the indexer

]]

[05.14.16 07:48:39] SEVERE: Caught an exception while invoking method 'run' on object 'BaselineUpdate'. Releasing locks.

Caused by java.lang.reflect.InvocationTargetException
sun.reflect.NativeMethodAccessorImpl invoke0 - null
Caused by com.endeca.soleng.eac.toolkit.exception.AppControlException
com.endeca.soleng.eac.toolkit.script.Script runBeanShellScript - Error executing valid BeanShell script.
Caused by com.endeca.soleng.eac.toolkit.exception.CasCommunicationException
com.endeca.eac.toolkit.component.cas.ContentAcquisitionServerComponent startBaselineCasCrawl - Error starting baseline crawl 'CBS-last-mile-crawl'.
Caused by com.endeca.cas.wsdl.CrawlNotFoundFault
sun.reflect.NativeConstructorAccessorImpl newInstance0 - null

[05.14.16 07:48:39] INFO: Released lock 'update_lock'.

APPLIES TO:

Oracle Commerce Platform - Version 10.2 and later
Information in this document applies to any platform.

CAUSE

Dimension Values are not rightly configured and imported into CAS components. Which results with above error during execution of baselineUpdate.sh on endeca APP.

SOLUTION

Note:  Exercise caution when directly modifying the versioned database schema.  For assistance identifying this problem and implementing the solution, please open an SR with Oracle Support.

1. Follow the steps in sequence

• Check you have Crawler available at http://localhost:8500 with the same name. e.g. <app>-last-mile-crawl.
• Generally this error occurs if you haven't initialized services.
• After deploying application to specified folder with help of deploy.xml
• Make sure you have <app>-data.xml and <app>-dimvals.xml available at location /test_data/basline folder.
• Goto application_directory/control and execute
  1. ./initialise_services.sh
  2. ./import_site.sh
  3. execute load_basline_test_data.sh
  4. ./baseline_update.sh
Your issue will be resolved.

ENDECA_MDEX_ROOT is required for this component,but eac.properties

Error

Caused by: An error occurred while trying to start the component: ENDECA_MDEX_ROOT is required for this component,but eac.properties does not contain a valid ENDECA_MDEX_ROOT definition.: ENDECA_MDEX_ROOT is required for this component,but eac.properties does not contain a valid ENDECA_MDEX_ROOT definition.

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:374)
at org.apache.axis.encoding.ser.BeanDeserializer.(BeanDeserializer.java:104)
at org.apache.axis.encoding.ser.BeanDeserializer.(BeanDeserializer.java:90)
at com.endeca.eac.client.EACFault.getDeserializer(EACFault.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)

Cause

This is problem is because one of MDEX properties in eac.properties file is missing or overwritten in installation.

SOLUTION

Please follow the steps below

1. Navigate to <endeca_installation> directory.
2. Search for eac.properties file.
3. Check each file with MDEX environment variable and update the property with correct path to MDEX with version directory. eg. C:/endeca/MDEX/11.1

Export / Import Endeca App between Different Environments

APPLIES TO:

Oracle Commerce Platform - Version 10.2 and later
Information in this document applies to any platform.

Export endeca Application Steps

1. Record Store - Click here to get more details. This Step can be optional if baseline triggers using /atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin Component

2. Dimension Value Id Manager - Navigate To CAS Bin folder and run following commands to export
CAS_Installation_Path/CAS/11.1.0/bin->./cas-cmd.[bat|sh] exportDimensionValueIdMappings -m <> -f /home/atg/dimvalid.csv

3. Endeca Pipeline - Copy Endeca_App/config/mdex folder

4. Editors Config - Copy Endeca_App/config/ifcr/configuration/tools/xmgr folder

5. Workbench Application - Navigate To Endeca Application control folder and run following commands to export content

       <>/control->./runcommand.[bat|sh] IFCR exportApplication

This would generate the zip/exploded folder.

6. Index-Config - Navigate To Endeca Application control folder and run following commands to export index-config.json

     <>/control->./index_config_cmd.[bat|sh] get-config -o all-File_Path/index-config.json

Import ENDECA APPLICATION STEPS

1. Record Store - Click here to get more details. This Step can be optional if baseline triggers using /atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin Component.

2. Dimension Value Id Manager - Run below command to import dimension values
/CAS/11.1.0/bin->./cas-cmd.[bat|sh] importDimensionValueIdMappings -m <> -f /home/atg/dimvalid.csv

3. Endeca Pipeline - Use the backup of <>/config/mdex folder and move pipeline.

4. Editors Config - Copy editors_config from step 5 and change host and post information based on environment under each json file and run the following command
         <>/control -> set_editors_config.[bat|sh]

5. Index-Config - Run below command to import index-config.json
<>/control->./index_config_cmd.[bat|sh] set-config -o all -f File_path/index-config.json

6. Endeca Baseline Update-  Run below Command

      <>/control -> ./baseline_update.[bat|sh]

7. Workbench Content - Run below command to import workbench content
        <>/control->./runcommand.sh IFCR importApplication <>

8. Promote Content - Run the following command to promote content from Authoring to Live. Promote content can be run the Workbench as well.

       <>/control -> ./promote_content.[bat|sh]

Important step to be performed in end

In the Last Do base line indexing from Store Dyn/admin from ProductCatalogSimpleIndexingAdmin component. This will help in creating exact dim mapping , data and dim vals for endeca schema. These files will not be visible for you to check the mappings.

Could not add user to DBS os group or TNS:no listner

Error

While starting the database getting error :

usr not in DBA os group 

TNS:no listner

Solution

Determine whether these groups and users already exist, and if necessary, to create them, follow these steps:

   1. To determine whether the oinstall group exists, enter the following command:

      # more /etc/oraInst.loc

      If the output of this command shows the oinstall group name, then the group already exists.

      If the oraInst.loc file exists, then the output from this command is similar to the following:

      inventory_loc=/u01/app/oraInventory

      inst_group=oinstall

      The inst_group parameter shows the name of the Oracle Inventory group, oinstall.

   2. To determine whether the dba group exists, enter the following command:

      # grep dba /etc/group

      If the output from this commands shows the dba group name, then the group already exists.

   3. If necessary, enter the following commands to create the oinstall and dba groups:

      # /usr/sbin/groupadd oinstall

      # /usr/sbin/groupadd dba

   4. To determine whether the oracle user exists and belongs to the correct groups, enter the following command:

      # id oracle

      If the oracle user exists, then this command displays information about the groups to which the user belongs. The output should be similar to the following, indicating that oinstall is the primary group and dba is a secondary group:

      uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)

   5. If necessary, complete one of the following actions:

          * If the oracle user exists, but its primary group is not oinstall or it is not a member of the dba group, then enter the following command:

              # /usr/sbin/usermod -g oinstall -G dba oracle

          * If the oracle user does not exist, enter the following command to create it:

              # /usr/sbin/useradd -g oinstall -G dba oracle

            This command creates the oracle user and specifies oinstall as the primary group and dba as the secondary group.

   6. Enter the following command to set the password of the oracle user:

      # passwd oracle

AssemblerTools : com.endeca.store.exceptions.PathNotFoundException: No node found at path: [pages/]

Error

Unable to retrieve site definition for site id: /: com.endeca.store.exceptions.PathNotFoundException: No node found at path: [pages/]

APPLIES TO:

Oracle Commerce Platform - Version 10.2 and later
Information in this document applies to any platform.

Cause

Above problem occurs when there is a mismatch bewteen site id in endeca app and Site Id of store front.

Solution

Follow below steps to get rid of this problem :

1. Navigate to <Endeca_installation_path>/apps/<project_app>/config/import/pages.
2. Ediit _.json file present.
3. Check defaultSiteId property and match with Site Id in Store Front.
4. You can check Site Id of store in BCC or through ACC.
5. Match both ids and restart endeca services.

Not able to access User / Profile Repository in ACC

Error

java.io.StreamCorruptedException: unexpected block data
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1364)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)

APPLIES TO:

Oracle Commerce Platform - Version 9.4 and later
Information in this document applies to any platform.

Cause

This error is because your particular Repository in ACC is not able to find particular class to load the properties of the xml.


e.g when you click on Users or Orders in ACC, above error appears in front of you.

Solution

So the solution is , find out if there are any external classes used for your repository. In my case pci encryption was used to encrypt password in UserProfile.xml and ACC was not able to locate that class when loading the particular repository.

So Add following lines in pci encryption module :

Go to encryption module MANIFEST.MF file

Add line ATG-Client-Class-Path: bin/ config/config.jar lib/classes.jar

Build project wit ANT and restart the server.

Cheers!!!

How to migrate Production / Stage Endeca App to Local Environemnt ?

Data Migration Needs

As per ATG-Endeca Integration, All product catalog data gets stored in Endeca CAS record store during baseline process. Sometimes, these data needs to be migrated from Production to UAT or UAT to Dev.

APPLIES TO

Oracle Commerce Platform - Version 10.2 and later
Information in this document applies to any platform.

Solution

Export Data from Environment - Production

1. Identify record store names to be migrated.

Navigate to CAS Installation directory and list the record stores

/CAS/<>/bin->./component-manager-cmd.sh list-components

CRS_en_dimvals

CRS_en_data

CRS_en_schema

2. Identify if there are any active-transactions or not

/CAS/<>/bin->

./recordstore-cmd.sh list-active-transactions -a CRS_en_dimvals

./recordstore-cmd.sh list-active-transactions -a CRS_en_data

./recordstore-cmd.sh list-active-transactions -a CRS_en_schema

3. Fetch the current record store configuration

/CAS/<>/bin->

./recordstore-cmd.sh get-configuration -a CRS_en_dimvals -f CRS_en_dimvals.xml

./recordstore-cmd.sh get-configuration -a CRS_en_data -f CRS_en_data.xml

./recordstore-cmd.sh get-configuration -a CRS_en_schema -f CRS_en_schema.xml

4. Read data from record store and write into XML files

/CAS/<>/bin-> 

./recordstore-cmd.sh read-baseline -a CRS_en_dimvals -f data_CRS_en_dimvals.xml

./recordstore-cmd.sh read-baseline -a CRS_en_schema -f data_CRS_en_schema.xml

./recordstore-cmd.sh read-baseline -a CRS_en_data -f data_CRS_en_data.xml

Import Data with schema to enviornment -UAT

1. Set the CAS record Store Configuration using step 3 during import process

/CAS/<>/bin-> 

recordstore-cmd.bat set-configuration -a CRS_en_dimvals -f CRS_en_dimvals.xml

recordstore-cmd.bat set-configuration -a CRS_en_schema -f CRS_en_schema.xml

recordstore-cmd.bat set-configuration -a CRS_en_data -f CRS_en_data.xml

2. Use step 4 data and write record into record store 

/CAS/<>/bin-> 

recordstore-cmd.bat write -a CRS_en_dimvals -f data_CRS_en_dimvals.xml

recordstore-cmd.bat write -a CRS_en_schema -f data_CRS_en_schema.xml

recordstore-cmd.bat write -a CRS_en_data -f data_CRS_en_data.xml

Note :- CRS_en_schema is not used for CAS based Indexing after 11.1 version.