loginusertrack

Login user track is a TYPO3 extension that records Frontend user activities. It records when user last logged in, how many pages he visited and what pages he visited. The extension was originally written by Kasper but was not updated for a long time. Netcreators needed additional functionality to this extension, so I took it, added that functionality and rewrote the other 50% of extension to use modern TYPO3 API. I also changed XCLASS to hook, which is always better. In addition Frontend handling is very efficient and does not slow down TYPO3.

Current update fixes a minor but very annoying bug with time units (seconds, minutes, etc) not visible in Backend statistics module.

Comments

The change in comments is significant. It is a result of my investgation of substituteMarkerArrayCached problems. Now comments does not use this function. It means it will run faster and will produce less data in the database. That gives another performance increase to TYPO3.

The obvious solution that come to the mind is to make an extension that will modify the form to include Frontend user data. Yes, it is possible.

But what I like about TYPO3 is that lots of things are possible without PHP programming. And adding Frontend user data to mail form is possible with a simple TypoScript. Here it is:

01: tt_content.mailform = COA_INT
02: tt_content.mailform.20.hiddenFields {
03:    name = TEXT
04:    name.data = TSFE:fe_user|user|username
05:    email = TEXT
06:    email.data = TSFE:fe_user|user|email
07: }

(This code assumes that you use CSS styled content, which you definitely should use now!)

Let's start with line 2. It adds hidden fields to the form. The "hiddenFields" property is an array of content objects. Name of the object becomes name of the form field. Value of the object becomes value of the field. In lines 3 to 4 and 5 to 6 I created two fields. I used "data" stdWrap property to fetch Frontend user data. If you do not know what is stdWrap or "data", you can read about it in my article about stdWrap.

How does it look like in HTML? Here is a fragment:

<input type="hidden" name="fromname" value="test01" />
<input type="hidden" name="frommail" value="test@example.com" />

What about line 1? Why is it there?

The answer is simple: you need to make form non-cacheable to avoid caching of the first user who visits this page. You need unique user name and e-mail.

What if user is not logged in?

If user is not logged in, you can use TypoScript conditions to add normal fields to the form.

Extension or TypoScript?

After making this part of code I remembered something. Several times I saw that people make extensions when several lines of TypoScript can do the same work. It reminds me that the first obvious solution is not always the best. There should be no rush with implementation. Careful thinking always produces much better result.

Did I already pronounced this before? Oh, yes. It was mentioned twice: once in my "Background thinking" article and second time during my "Personal productivity" session in TYPO3 user group in the Netherlands in July 2008.

What do you think about it?

Enjoy!

I'd like to ask you nominate me if you think I deserve it. It is easy as filling the form and clicking "Nominate" button. Btw, you can win iPod Nano if you nominate someone.

Thank you in advance!

P.S. My name is Dmitry Dulepov.

Yesterday, July 17, near 11:00 two men, one of them said to be a teacher at Latvian Sport Pedagogical Academy, throw a small cat to their dog. The dog broke kitten's stomach.

Look in his eyes! He probably came to people hoping to play with them. Small cats do this... But they threw him to the big dog as a piece of meat.

Police was called by people passing by. When arrived, police made a fine to dog's owner for walking with dog without muzzle. They did not want to register cruelty to the kitten.

Pure creature was in agony all time while transferred to animal hospital by rescue team. Doctors were not able to save it. They have to give the kitten deathly injection to stop his suffering.

State authorities will write an official letter to police asking to open a crime case. It is not clear if police will do it or not.

Latvia calls itself "European country". So far only one was was prosecuted for cruelty to animals in Latvia. All others avoided prosecution.

The answer is pretty simple. Make a new "hide in menu" page. Insert Sitemap there. Logout from Backend. Enter that new page address in the browser bar.

Notice that logging out is essential. RealURL will not cache anything if you are logged in to Backend.

This procedure will fill cache with page addresses. But it will not create cached entries for stuff like tt_news. So this tip is good if you have mostly pages.

As you remember there are four “substitute” functions for use with TYPO3 templates:

• substituteMarker
  This function substitutes a single marker
• substituteMarkerArray
  This function does the same as above but for many markers in the array
• substituteSubpart
  Substitutes a single subpart
• substituteMarkerArrayCached
  Our today's case.

The first two functions substitute marker and marker array. The third substitutes template subpart. The obvious missing function is the one to substitute subpart array.

substituteMarkerArrayCached takes several arguments, three first are most interesting to us. It takes template, marker array and subpart array. It means that this function works almost like the function we just found missing. In other words, it can substitute subpart array:

$content = $this->cObj->substituteMarkerArrayCached($template, array(), $subPartArray);

But there is a catch.

The catch is in the “Cached” word in the function name. This function caches its results and tries to reuse them next time. Caching happens in the cache_hash table.

It does not look bad from the first glance. It may even speed up the web site.

Does it really?

Imagine a web site with articles. Let's say 10000 articles. Each article has comments. Suppose that average number of comments is 10 per article. So, if comments use substituteMarkerArrayCached, it would lead to 100000 records in cache_hash table. Now imagine of each comment requires two calls to substituteMarkerArrayCached...

You can say: “Big deal! Disks are large and cheap now!”. Yes, they are. But it is not a disk space. It is MySQL who will suffer.

MySQL works very well with indexes. If it can locate a record using index, the speed can be fantastic. Autoincrement fields make a good index. This is a typical case for most tables in TYPO3. But cache_hash uses MD5 value as index. MD5 values look like random data for MySQL. They do not make good database indexes. So you can be easily out of luck with this table when think about indexes.

Another problem lies in a large number of records in this table. MySQL uses indexes only if it believes that it can save time by using them. If MySQL thinks that using index will not result in much performance gain, it will revert to full table scan. Imagine ful table scan on 100000 row table. Wouldn't non-cached version of substituteMarkerArrayCached be faster for a two entries in $subPartArray?

Unless you have really huge subpart and marker array, I recommend to avoid using substituteMarkerArrayCached at all. It can make performance better for really large substitutions but not for smaller ones. PHP code often runs faster than a single database query (especially if query goes over network).

You can replace a call to substituteMarkerArrayCached with the following code:

$content = $this->cObj->substituteMarkerArray($template, $markers);
foreach ($subParts as $subPart => $subContent) {
    $content = $this->cObj->substituteSubpart($content, $subPart, $subContent);
}

This is how comments extension is going to work starting from the next version.

Summary

substituteMarkerArrayCached is bad because it stores results in and fetches results from cache_hash table. This causes lots of records and slower execution for a simple substitutions. So do not this function unless your substitutions contain a lot of data.

Let's have an overview of TYPO3 templates and functions to handle them.

TYPO3 template basics

Templates in TYPO3 are normal HTML files. These files contain special markers that TYPO3 can replace with data. Markers typically are words in CAPS surrounded by three hash marks. For example: ###LINK###.

There are two types  of markers: single markers and subpart markers. Single markers is a single piece of data, it is replaced as is. Subpart markers surround parts of the HTML code. Here is an example:

<!-- ###LIST### begin -->
Repeating elements:
<ul>
<!-- ###REPEATING### begin -->
<li>Iteration ###NUMBER###</li>
<!-- ###REPEATING### end -->
</ul>
<!-- ###LIST### end -->

Here ###LIST### and ###REPEATING### are subpart markers. Extension can get a subpart and replace it as whole. Often it is used for repeating elements of for subparts that should be removed under certain conditions. “begin” and “end” are not necessary but conventional.

###NUMBER### is a single marker.

Template functions in TYPO3 Frontend

TYPO3 has functions to work with templates. Extension authors may use these function to process markers.

The functions are located in class tslib_cObj and available in Frontend plugins through $this->cObj. Here is the list:

• getSubpart
  Obtains a subpart from template
• substituteMarker
  This function substitutes a single marker. Just that.
• substituteMarkerArray
  This function does the same as above but for many markers in the array
• substituteSubpart
  Substitutes a single subpart
• substituteMarkerArrayCached
  This is a bad function that you should not use. Read why.

As you see, there is no function to substitute subpart array. You have to do it in a loop for each subpart.

Using TYPO3 template functions

But let's see code example. Example assumes that HTML template is located in $this->templateCode.

function generateRepeatingContent() {
    // Get template
    $template = $this->cObj->getSubpart($this->templateCode, '###LIST###');
    // Get subpart template
    $subTemplate = $this-cObj->getSubpart($template, '###REPEATING###');
    // Loop to create repeating content
    $subPartContent = '';
    for ($i = 1; $i <= 10; $i++) {
        $subPartContent .= $this->cObj->substituteMarker($subTemplate,
            '###NUMBER###', $i);
    }
    // Substitute subpart
    return $this->cObj->substituteSubpart($template, '###REPEATING###',
        $subPartContent);
}

This example uses the same template as shown above.

Conclusion

This is how easy to use templates in TYPO3. So when you develop extension next time, you can make it better by providing templatable output for your extension.

I got a lot of requests for these slides and I promised to put them online. Here is the TYPO3 core presentation. It did not contain much text on the slides, so I made a QuickTime videos. Videos are available in two sizes:

• Small (256x192, 4.6MB)
• Medium (512x384, 9MB)

I must tell that this is my first experience in making video out of presentations, so I apologize for any problems in the video. People in TUG liked the presentation and I hope you all like the video.

This video will also be available through typo3.nl web site.

License & disclamer: anyone can freely copy and distribute these movies without modification (no cutting or extracts permitted). I tried to make facts as accurate as possible but you need to check them if you want to use them for commercial purposes. I am not going to be responsible for misuse of this material or any loses that come from it.

[Update, July 21, 2008] Now you can also see these slides on SlideShare but without audio.

It is not a big problem for the particular case described in that article. In the article this function will be used only for a logged-in user who views his own card. But readers may learn about the function and start using it in extensions without realizing what they do. That's bad. We had a problem some time ago in commerce extension, which disabled caching for the whole web site and made the site extremely sloooooooow and created huge load on the server. Bad. Too bad. I wish all writers make a bold warning when mentioning this function: dangerous!

Last week I made a performance review of yet another web site and yet again (again and again!) I found problems caused by misuse of set_no_cache(). It was found in three extensions (download, link list and one proprietary). It is typically the set_no_cache-case when people ask why their site has such a bad performance. If it only were in my power, I would go to TER and remove all extensions that use set_no_cache().

Don't use this function! It is for core only! It is not for you! Avoid at all costs!