Dromaeo benchmarking – IE11, FF R31, Chrome R37 and Canary

Besides Google’s Octane 2 there is another extensive benchmarking framework: Dromaeo.

I took it to the test for the usual suspects.

The good thing about Dromaeo is that it has dedicated tests for jQuery, very useful with OpenUI in mind :-)

And the results are… I ran a number of tests on my good-old i5-2520M@2.5Ghz. I ensured while running the test the machine was idle.

DromaeoFrameworks DromaeoCoreDOM DromaeoJslib

The tables list the number of runs/second for the different test dimension. From those I calculated the performance factors. So “Chrome vs. IE” has a performance factor 2,9 for the “DOM Attribute (Prototype)” dimension. So Chrome R37 is 2,9x faster than IE11 for this particular characteristic.

It is clear that IE11 overall is being surpassed by its rivals. Especially when looking at the relevant dimensions for Siebel Open UI (jQuery) Chrome is the overall winner. For jQuery DOM Query/selector operations Firefox and Chrome end up ex-aequo. More general the jQuery framework is better tailored for Chrome. The more all-purpose DOM operations are best tailored for Firefox (these entail direct DOM operation without using any framework).

On generic DOM traversal IE11 is tailing its rivals.

Comparing browsers for OpenUI – Octane 2 as a benchmark tool?

Recently I took Google Chrome Canary 64-bits (R39) to the test. It appeared that Google’s own disclaimer:

Be forewarned: it's designed for developers and early adopters, 
and can sometimes break down completely.

Should be taken seriously. Both Canary and the stable version (R37) should be able to run side-by-side. Even though both R37 and R39 were working flawlessly for normal browsing, my Siebel dedicated client would no longer run against R37. Uninstalling both releases and installing R37 fresh resolved the issue.

First thing I did: run Google’s own Octane2 benchmark against both browsers a couple of times. I made sure there were no other CPU intensive tasks running.

Interestingly enough R39 falls behind R37 overall.

Octane is a an open source initiative from Google. The dimensions against which Octane tests browsers are shown on the left-hand side. In the middle section contains the rough results for IE11, FF31, Chrome R37 and R39 are shown. On right-hand side performance factors are calculated.

BrowserOctane2results

R37 overall is a factor 1,1 faster than R39. On certain dimensions the differences are more than a little.

The red highlighted rows have in my personal view are important dimensions for Siebel OpenUI performance. The dimensions are pretty well explained here.

For example “Splay” and “SplayLatency” are relevant to the OpenUI framework. OpenUI depends on JQuery and executes fast amounts of script. So object creation, destruction and an efficient garbage collection process are key.

  • Splay
    Data manipulation benchmark that deals with splay trees and exercises the automatic memory management subsystem. Main focus: Fast object creation, destruction.
  • SplayLatency
    The Splay test stresses the Garbage Collection subsystem of a VM. SplayLatency instruments the existing Splay code with frequent measurement checkpoints. A long pause between checkpoints is an indication of high latency in the GC. This test measures the frequency of latency pauses, classifies them into buckets and penalizes frequent long pauses with a low score. Main focus: Garbage Collection latency.

Put it together in a visual representation…

OctaneGraphOUI

What I’m personally particularly interested in is whether Octane2 could be a valid benchmark for Siebel OpenUI in particular. Octane2 is developed by Google, so will Chrome prevail by definition over other browsers? There are other benchmarks such as Sunspider (see previous post) and Dromaeo. The nice thing about Dromaeo is that it splits up in certain categories of tests focussed e.g. on CSS selection and JQuery (used quite a bit in OUI as we know). I did not used Dromaeo so far, on the list. Dromaeo tests are more elaborate and take more time to complete.

So let’s loop back to my previous post where I build a number of increasingly heavy Siebel OpenUI views. I ran a new set of timing tests using the described mechanism.

You will see two rows (collapsed and expanded). For now – just neglect this. Will spent a separate post on my challenges to improve complex views – rather than splitting the applets across different views ;-)

Collapsedexpanded

For example these test tell that Chrome R37 is on average 1,2x faster than R39. That compares to the Octane2 results. Especially if you relate it to Siebel OpenUI relevant test dimensions:

OpenUISpecificOctane2

Also the Chrome vs FF (1,4x) compares pretty well to the timings measured in my OpenUI stress test views.

But to comparing to IE it would be a very bad indicator… Where the OpenUI performance factors are 3,4x for FF vs IE and 4,3x for Chrome vs IE – Octane would respectively indicate a factor 1,8x and 2,5x.

What is clear is that the performance breakdown on IE11 follows an exponential function, where as for FF and Chrome it’s a more linear curve. Which was made clear in my previous post already.

In a next post I will dive a bit deeper into Dromaeo as well to continue my quest to improve rendering large views in OpenUI.

 

Why Oracle Application Testing Suite for Siebel OpenUI stands out

A nice webcast about OATS 12.4 features with regards to Siebel OpenUI.

OATS 12.4 (the most recent incarnation today is 12.4.0.2) supports Siebel OpenUI.

OATS 12.4 comes with a specialized Open UI Function Library which is a fully documented API which originated from Oracle’s Siebel QA team.

The OpenUI Function Library is completely documented in the OpenScript Programmer’s Reference which you can grab at OTN. Scroll down to the bottom to find the link “Download Oracle Application Testing Suite product documentation set”.

One word of care: The OpenUI Function Library should ideally not be an alternative for record-and-playback. It makes most sense to record (re-usable) modular scripts but the function library could be used to enhance these scripts. Building OpenScripts from scratch – you can imagine – will be a time-consuming activity and a maintenance night-mare.

Effectively rewriting Siebel Predefined Queries for performance

Ever had to deal with PDQs which required to fetch based on date functions such as Year-to-date or Month-to-date (or any related…)?

I came across an implementation where a customer became very creative trying to resolve this. But the end-result was a terrible performance. Why? Because the PDQ could not be completely be executed as SQL.

A generic implementation flaw in queries written by Siebel configurators or business analysts: misusing calculated fields to be used in e.g. search Expressions and PDQs. It can (or will) hammer performance. A lot.

For example:

Provide me all the Opportunities YTD. This was the original PDQ:

"[Due Date] <= Today() AND JulianYear(Today())=JulianYear([Due Date])"

It does the job. But the SQL’s WHERE clause would only include the [Due Date] <= Today() clause. Assume you have some 15 years of Opportunities. It would fetch all records. Only in-memory the object manager would be able to further filter based on the condition JulianYear(Today())=JulianYear([Due Date]). You can imagine how resource-extensive this would be. Not to imagine the end-user performance perceived.

Similar constructions for Month-to-date and Quarter-to-date queries.

How to circumvent this?

Goal would be to have ProfileAttributes available throughout the application which would carry values such as:

1st day of the year - "01/01/2014"
last day of the year - "31/12/2014"
1st day of the month: - "01/08/2014"
last day of the month: - "31/08/2014"

Well, you get the point.

To realize this you can easily configure a number of fields on the “Personalization Profile” business components. The nice feature of this business component is that all fields are loaded for every session immediately after login. And those fields, well, become Profile Attributes. Typically the “Personalization Profile” business component consist out fields which can be joined toward the Party record for the user logging in (can be an Employee, but can be also a Portal user). But you can also create Calculated Fields. And that will be of great help. Consider the Calculated Fields below (you can grab the complete.xls here).

PersonalizationProfile

The “green” onces are the interesting profile attributes. The white onces are just supporting field to make the calculated fields somewhat readable.

Now let’s rewrite the PDQ from the example.

"[Due Date] <= Today() AND JulianYear(Today())=JulianYear([Due Date])"

The optimized version would become:

[Due Date] > GetProfileAttr("Year Start") AND [Due Date]) < Today()

The optimized PDQ would translate completely into a more enjoyable SQL WHERE clause. It will no longer have to fetch unneccesary data. Let the database take care of this. And of course ensure an appropriate index exist for an efficient execution plan :-)

 

A closer view on Siebel OpenUI performance, the Firefox – IE11 battle

In a recent previous post I already concluded that IE11 is not an ideal browser for Open UI. Which obviously is shame because still the most widely used browsers within organizations. In private life though people tend to use different browsers. But even though the decline for IE is ongoing it will take time before IE is replaced in workplace environments.

630px-Usage_share_of_web_browsers_(Source_StatCounter).svg[1]

Organizations tend to move towards Firefox rather than Chrome. Most likely because Chrome being a Google product, too risky? Firefox as Open source is assumed a safer bet. But this goes at a very slow pace. We should not expect that to change dramatically over the coming years.

So, we have to deal with IE. That is why I created in Siebel OUI as couple of “test” views. Large views which have increasingly more applets and controls. These views are build on top of a VBC in order to cancel out any false-positives from the test results (e.g. to measure only rendering, not queries and fetching).

Taken for granted that I created huge views which you will never (?) see in a typical Siebel environment. But I wanted to take a somewhat scientific approach here to find a possible pattern. View 1 and 2 could be considered still realistic in terms of number of controls in certain customer scenarios – though 2 could be considered pretty outrageous.

  • View 1 (indicative: 40 controls in total)
    • One Form applet with 20 controls
    • One List applet with 20 list controls
  • View 2 (indicative: 240 controls in total)
    • One Form applet with 20 controls
    • One Form applet with 100 controls
    • One List applet with 20 list controls
    • One List applet with 100 list controls
  • View 3 (indicative: 440 controls in total)
    • One Form applet with 20 controls
    • Two Form applet with 100 controls
    • One List applet with 20 list controls
    • Two List applet with 100 list controls
  • View 4 (indicative: 640 controls in total)
    • One Form applet with 20 controls
    • Three Form applet with 100 controls
    • One List applet with 20 list controls
    • Three List applet with 100 list controls

If you want to grab the .sif file – go ahead :-) It contains the VBC, Applets, Views and Screen. You only need to register the screen in the application of your wish.

Ok, what have been my observations? Firefox is the winner with IE11 on the other end of the spectrum.

What would be your leading design practice: design & keep your views small ;-) No, really – the should be kept small if you want a view to render sub-second. And prevent at all times to enable too many list columns. The common reason “we need it for export” should be reconsidered. Where that might work in HI – in OpenUI it has severer implications. The JQGridRenderer has a noticeable overhead.

Do not forget that the timings below (in milliseconds) are measured between “PreLoad” and “PostLoad”. Those are JScript events of the OpenUI API. But these timings exclude further rendering time (such as CSS).

IEFF

FF IE11

Siebel IP2013 Patchset 10 has been released

Patch Set 10 has been released. Refer to:

Siebel Patchset Installation Guides for Siebel CRM 8.1.1.11 / 8.2.2.4 IP2013 (Doc ID 1614310.1)

Patch numbers:

19295121 (8.1.1.11.10)
19295120 (8.2.2.4.10)

A big patch set with 52 addressed issues. POO

For the manifest administration issues (18048557) a separate tool has been included which will rectify orphans.

Issue Addressed Bug ID Product Area Category
OPEN UI – SERVICE ACCOUNT FIELD FOR ASSET CANNOT BE UPDATED 18829026 Siebel Apps – Customer Order Management Open UI
MISSING POPUP APPLET IN EDIT PROMOTION USER INTERFACE AFTER ACR 652 18816318 Siebel Apps – Customer Order Management Product Promotions
MEMORY LEAK ADD-/REMOVING ITEMS IN PRODUCT DOMAIN OF PRODUCT PROMOTION PRODUCT 18750636 Siebel Apps – Customer Order Management Product Promotions
ATTRIBUTE ADJUSTMENT CAUSING ERROR ON ORDER VERIFY 18865198 Siebel Apps – Customer Order Management Quotes & Orders
SIEBEL LEAD IMPORT – EXACT MATCH NOT WORKING. 18792760 Siebel Apps – Marketing Mktg/Leads
ADDS A NEW OBJECT MANAGER FOR PARTNER PORTAL FOR VERSION 8.1.1.11 UPGRADES 18748251 Siebel Apps – PRM PRM Partner Portal
ERROR OCCURS USING CHILD SURVIVORSHIP WHEN CHILD ASSOCIATED TO A GRANDCHILD. 18750579 Siebel Apps – UCM Universal Customer Master
OPEN UI-SAVE TEMPLATE BUTTON IS CREATING EXTRA WHITE SPACES ON TOP OF THE TEMPLATE 18622823 Siebel Apps -Marketing Mktg/Campaign Mgmt
CURSOR FOCUS CHANGE TO CHAT PANEL WHEN CHAT MESSAGE FOR CHAT ON HOLD 18797497 Siebel Core – Common Components CRM Chat
NEED WINDOWS 2008 OS SUPPORT FOR SIEBEL CCA CTI DRIVER 18959377 Siebel Core – Common Components CTI
OPEN UI SMARTSCRIPT WITH RADIOBUTTON WITH SPACE OR SINGLEQUOTE HANGS 18425065 Siebel Core – Common Components CTI
PRINT PREVIEW POPS UP ERROR ‘THIS APPLET HAS NOT BEEN CONFIGURED FOR PRINTING’ 17304406 Siebel Core – Common Components Email Issues
IHELP SHOWING SERVER BUSY MESSAGE 19034422 Siebel Core – Common Components iHelp
USER CANNOT SELECT RADIO BUTTONS IN OPEN UI SMARTSCRIPTS FOR 8.1.1.10 18937087 Siebel Core – Common Components SmartScript
SMARTSCRIPTS ARE MISALIGNED 19192925 Siebel Core – Common Components SmartScript
DUPLICATE RECORDS ARE PRESENT IN MANIFEST ADMIN VIEWS AFTER REPOSITORY EXPORT/IMPORT 18048557 Siebel Core – DB Deployment and Configuration IRM
INCORRECT CONTENT LENGTH AND CONTENT –TYPE IN HTTP RESPONSE ON IPLANET 18797210 Siebel Core – EAI Integration – Transports
GC_HEAPSIZE BEING LOGGED EVEN IF SIEBEL_LOG_EVENTS IS 0 18618229 Siebel Core – Server BizLogic Script Config – Scripting
 IIS APPLICATION POOL CRASH 18750734 Siebel Core – Server Infrastructure Server Infrastructure
SBL-SRB-00047: COULD NOT ROUTE MESSAGE TO XXX WITH REGISTERED KEY 18551942 Siebel Core – Server Infrastructure Server Infrastructure
UNABLE TO SUBMIT GENTRIG JOB IN GUI MODE 16421088 Siebel Core – Server Infrastructure Server Infrastructure
SBL-DAT-00309 ERROR OCCURS DURING TASK ENTRY 18115019 Siebel Core – Server OM Frwks Business Process Framework
ONLY ONE VIRTUAL DIRECTORY IS CREATED AFTER CONFIGURATION FOR NON-ENU LANGUAGES 18815122 Siebel Engineering – Installer Install – Configuration
EXPIRING LOYALTY POINTS WITH REDEMPTION CONCURRENCY FUNCTIONALITY MISSING TXN_ID 18802469 Siebel Loyalty Redemptions
USERS CANNOT SELECT AN ADDRESS FROM A PICKLIST IF THERE ARE EXACTLY TWO ADDRESSES 19233523 Siebel Mobile Applications Mobile Framework
DB INITIALIZATION FAILURE DURING TOOLS LOCAL DATABASE SETUP 19060741 Siebel Server Remote Remote
SIEBELANYWHERE PATCH UPGRADE: INSTALLER CRASHES IF MIGRATE OPTION IS USED 18767357 Siebel Server Remote Remote
UNABLE TO QUERY BY ROOM BLOCK NAME IN QUOTE DETAIL APPLET 18995620 Siebel Travel and Transportation Diary
SERVICE CHARGE TYPE FIELD ON FUNCTION SPACE LINE ITEM DEFAULTS TO POST 19128881 Siebel Travel and Transportation Functions
FUNCTIONS COPY TO INCORRECT DAYS FOR MULTI-DAY FUNCTIONS 18995696 Siebel Travel and Transportation Functions
NEGOTIATED ROOM RENTAL DOES NOT COPY WHEN FUNCTION IS COPIED 18995438 Siebel Travel and Transportation Functions
EXPANDED SUB FUNCTIONS ARE ALSO COPIED AS PARENT FUNCTIONS 18995300 Siebel Travel and Transportation Functions
SIEBEL HIGH-INTERACTIVITY OBJECT MANAGER CRAHES UNDER HIGH LOAD ORDER MANAGEMENT SCENARIOS 18998841 Siebel UI Framework AX/HI Web UI
ORDER FIELD LINE VALUES ARE COPIED WHEN THE EXPAND METHOD IS INVOKED 18228643 Siebel UI Framework AX/HI Web UI
WITH OPEN UI, BROWSER SCRIPT IS CALLED TWICE IN FORM APPLETS 18147389 Siebel UI Framework Open UI
MANIFEST FILE IS DOWNLOADED DESPITE THE INACTIVE FLAG BEING SET TO TRUE 19178747 Siebel UI Framework Open UI
ORPHAN RECORDS IN S_UI_OBJ_EXP_FL 19244417 Siebel UI Framework Open UI
REQUIRED FIELD ERROR MESSAGE OCCURS WHEN TASKS ARE CANCELLED 18990998 Siebel UI Framework Open UI
COUNT (*) SQL CALLED IN OPEN UI: POOR PERFORMANCE OCCURS WHEN CREATING NEW RECORDS IN PICK APPLETS 18922789 Siebel UI Framework Open UI
OPEN UI: PICKLIST BUTTONS NOT ENABLED/IN CORRECT FORMAT 18841731 Siebel UI Framework Open UI
OPEN UI: APPLICATION HANGS IF APPLICATION_PRENAVIGATE HAS RAISEERRORTEXT CALL 18733979 Siebel UI Framework Open UI
OPEN UI SMARTSCRIPT WITH RADIOBUTTON WITH SPACE OR SINGLEQUOTE HANGS 18648505 Siebel UI Framework Open UI
OPEN UI: EMAIL ADDRESS DOES NOT APPEAR AS A HYPERLINK 18629177 Siebel UI Framework Open UI
OPEN UI BROWSER SCRIPT SETPROPERTY FOR READONLY IS NOT WORKING PROPERLY 18550190 Siebel UI Framework Open UI
OPEN UI: ACCESSIBILITY FEATURES CTRL+ALT+T NOT WORKING 18512296 Siebel UI Framework Open UI
OPEN UI: WHEN TABBING WITHIN FORMS/LISTS, FOCUS SHOULD NOT LEAVE FORM/LIST 18484096 Siebel UI Framework Open UI
OPEN UI: DROP-DOWN LISTS DO NOT RENDER WHEN CONFIGURED WHEN HTML ATTRIBUTES PROPERTY IS USED 18405162 Siebel UI Framework Open UI
OPEN UI: AN ERROR MESSAGE OCCURS THE FIRST TIME A TEMPLATE IS ATTACHED 17174123 Siebel UI Framework Open UI
OPEN UI: ICONS DISAPPEAR ON THE ACCCOUNT PICK APPLET 19176963 Siebel UI Framework Open UI
OPEN UI: INCORRECT OPTION FOR ASSET TO AGREEMENT 19164623 Siebel UI Framework Open UI
OPEN UI: INCORRECT MANDATORY FIELD ERROR MESSAGE DISPLAYS 19164013 Siebel UI Framework Open UI
OPEN UI: IFRAME NOT ABLE TO COMPLETE GOTOVIEW CALL FROM WORKFLOW 19154449 Siebel UI Framework Open UI