Last night I decided to let MySQL think about that query ... It seemed to have completed calculations in the morning. Not surprisingly it has returned the very same data like did the MS SQL - The multiplication of the results. But it has taken 6604156 milliseconds (~6604 seconds) to obtain the result. The same result that MS SQL EXPRESS returned in 274709 milliseconds (~275 seconds).
No further comments.
Case closed.
Hi again.
Today I made a test - transferred the data into MS SQL Server 2005 Express SP 2.0.
Guess What ? :)
The situation is almost the same ! Wondering why the result was messed, the execution time was arround '274709' - I guess this is the "ms" - "duration" column in the MS SQL Server Profiler. Which is arround 5 minutes.
The mess I am talking about is the following:
The situation is that in of the tables I have about 120k records, which should be displayed, on the other one I have about 5k records. Insted of showing "120000" for "di_count", and "5000" for "ei_count", in the result I got '594163480' (which is the number of rows in one of the left joined tables multiplied by the number of rows in the other one) for each of them - the same figure for the two columns. What is that ?
I just want to make a select from one table, than join second table, then join third table to the FIRST one, then join the last table AGAIN to THE FIRST one. Why is it joining the result alltogether?
Anyway .. using UNION, VIEW and work with the VIEW solves the problem, but I was just wondering why this happens ...
I j ust can't help showing you these two screens:
For the MSSQL Profiler:
And for the MySQL (note: the execution is still not finished!):
I have no comment!
Ok, this is will be about MySQL once again.
I had that problem. Let's assume we have 3 tables:
1. smetki
2. di_invoices
3. dk_invoices
4. ei_invoices
Each of the XX_invoices has a "foreign_key" to PK of "smetki". I am using MyISAM.
If I perform the following 3 queries separately:
---------------------------------------------------------
select
s.smetka_id, s.created_on,
count(di.invoice_id) as di_count
from
smetki as s
left outer join di_invoices as di on (s.smetka_id = di.smetka_id)
where
s.created_on between '2007-12-10' and '2007-12-15'
group by
s.smetka_id
-----------------------------------------------------------
select
s.smetka_id, s.created_on,
count(dk.invoice_id) as dk_count
from
smetki as s
left outer join dk_invoices as dk on (s.smetka_id = dk.smetka_id)
where
s.created_on between '2007-12-10' and '2007-12-15'
group by
s.smetka_id
------------------------------------------------------------
select
s.smetka_id, s.created_on,
count(ei.invoice_id) as ei_count
from
smetki as s
left outer join ei_invoices as ei on (s.smetka_id = ei.smetki_id)
where
s.created_on between '2007-12-10' and '2007-12-15'
group by
s.smetka_id
------------------------------------------------------------
I have no problem, I have proper indexes and I got the results for less then 200 ms from each query.
But I need the 3 results into 1 single resultset so I decided to make a join like that:
select
s.smetka_id, s.created_on,
count(di.invoice_id) as di_count,
count(dk.invoice_id) as dk_count,
count(ei.invoice_id) as ei_count
from
smetki as s
left outer join di_invoices as di on (s.smetka_id = di.smetka_id)
left outer join dk_cor_invoices as dk on (s.smetka_id = dk.smetka_id)
left outer join ei_invoices as ei on (s.smetka_id = ei.smetki_id)
where
s.created_on between '2007-12-10' and '2007-12-15'
group by
s.smetka_id
Now I have query which executes more than 15 minutes !! I couldn't wait to the end and killed it. There must be something wrong somewhere?
I sould notice that for the given period there are only 6 rows in the "smetki" table, and different number of rows in each xx_invoices table that correspond to the whole resultset from "smetki". I have to mention that there is one intersection between two of the XX_invoices tables where there are about 120k rows for one "smetka_id", and for the same "smetka_id" there are about 5k rows in the other.
Also - there are no other queries running by the time of measuring the performance.
What could be the solution. A friend of mines helped me. Make a UNION he said. So I did it like that:
(select
s.smetka_id, s.created_on,
count(di.invoice_id) as di_count, 0 as dk_count, 0 as ei_count
from
smetki as s
left outer join di_invoices as di on (s.smetka_id = di.smetka_id)
where
s.created_on between '2007-12-10' and '2007-12-15'
group by
s.smetka_id)
UNION
(select
s.smetka_id, s.created_on,
0 as di_count, count(dk.invoice_id) as dk_count, 0 as ei_count
from
smetki as s
left outer join dk_invoices as dk on (s.smetka_id = dk.smetka_id)
where
s.created_on between '2007-12-10' and '2007-12-15'
group by
s.smetka_id)
UNION
(select
s.smetka_id, s.created_on, 0 as di_count, 0 as ei_count,
count(ei.invoice_id) as ei_count
from
smetki as s
left outer join ei_invoices as ei on (s.smetka_id = ei.smetki_id)
where
s.created_on between '2007-12-10' and '2007-12-15'
group by
s.smetka_id )
OK, but now I have 3 rows for each single "smetka_id" - one row for the results from each table. And then we decided to make a VIEW over that UNION.
Gues what. Now I have the simple query:
select smetka_id, sum(di_count), sum(dk_count), sum(ei_count) from v_test group by smetka_id
Which returns me the desired result for less than a second !
I hope this would help your daily work with MySQL and SQL in general.
Lot friends of mines were telling me that I should watch that movie (also in wikipedia). I recently had the chance! And it really worth!
It's a documentary film about climate change, specifically global warming, presented by former United States Vice President Al Gore. There are lots of numbers and facts that can make you reconsider some aspects of your living.
So I urge you to watch this movie!
Hello again ;)
I ran into a tricky situation last couple of days. I have some big tables with varchar fields where I have to JOIN ON varchar fields. The problem does not concern only the JOIN - it is a general problem using varchar / text fields used to be searched one way or another.
So - the situation: We have one table with about 1M rows (relatively small amount data for each row ~90 bytes) on one hand. And another table with records to be matched - about 100k rows (relatively medium amount of data for each row ~ 500 bytes). We do a SELECT which JOINs the two tables based on VARCHAR columns. Do not ask "why?" - This is the business logic; it cannot be otherwise (at least in the beginning). If we index the columns we will got the results fast! What is fast? Less than 50 ms to get around 200 joined rows. Well, OK, you got me - it depends strongly on the hardware configuration and MySQL settings. Let's assume this is our limit. I was very happy to discover that it works that fast. The speed is critical, because we will need to join around 100k rows.
Where is the problem then? Well, it happened that the first table is populated with another several hundred thousand records and it became with about 1.9M rows. And? And, it appeared that the MySQL should copy the temp table on the hard drive. As you can guess - the result is: 204 joined rows (not 204k, but only two hundred and four) for ..... 417 seconds. What about 100k rows? Killing me.
What can we do about that? Well, I know (the business logic predicts) that the first table - that with about 2M rows would never have more than 4 million rows. I also noticed that partitioning is only partially supported in the beta (RC yet) of MySQL 5.1.x, but the recommended for production environment is 5.0.x which does not support partitioning. So, I discovered the great options MAX_ROWS and AVG_ROW_LENGTH. Which does what - it says to the MySQL engine, that this table would never have more than MAX_ROWS tows, and the average row length would be AVG_ROW_LENGTH. You can find the average row length by executing the command SHOW TABLE STATUS FROM [DB_NAME] LIKE 'TBL_NAME'. So, if you know, if your business logic predicts that you will have a large table, BUT this table will have relatively known content (like it will have not more than 4M rows), and you know the average row length, you can set the MAX_ROWS and AVG_ROW_LENGTH for that table. The AVG_ROW_LENGTH is required if you have blob/text/varchar fields.
That will cause the MySQL engine to optimize the space required for that table, also optimizing the myisam_data_pointer_size, which sets the number of bytes used for internal row pointers. Yes - it works only with MyISAM tables. But you will be surprised. After I set the MAX_ROWS and AVG_ROW_LENGTH on that table with the 2M records - I got a surprising results - 90k joined rows for ..... 47 seconds !!
I think I will use these options often in the future, what about you?
Guess what! The little tricks of skype does not have an end ... The Linux users of Skype recently discovered that it is reading the /etc/passwd and firefox profile files. COME ON? Do you continue using Skype?
The original post is here: http://forum.skype.com/index.php?showtopic=95261
And some explanations can be found here: http://www.oreillynet.com/sysadmin/blog/2007/08/a_close_look_at_questionable_a.html
But I seriously consider using Skype ...
Do you?
Did you have a task for monitoring files?
The FileSystemWatcher class has a great features, but also some outages. The problem is that creating/copying/moving files arround is hard taks and the watcher raises couple of events.
The most important events in my case were "created" and "changed". The problem is that the "Created" event is fired when the file is started being created, not when finished being created. After the created, there would several "changed" events depending on the file size. So how could we possibly know when the file is realy created, full and ready to work with it?
There are several posts and examples arround the net like:
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2123580&SiteID=1&mode=1
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=659823&SiteID=1
What is my approach?
After catching any event - you just stop monitoring for FSWEvents and start a timer. On every tick you try to open the file for reading. If you cannot - increase the timer interval. You can also add a code to monitor the total time elapsed since the first try to open the file, but the provided code does not do that.
So here it is - one of the approached to more correctly monitor for files:
using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Security;
using System.Timers;namespace MonitorTest
{
public class Program
{
private FileSystemWatcher _watch;
private string _watchPath = @"d:\temp\";
private string _watchFile = "SomeFile.txt"
private Timer _timer;
private DateTime _started;
private TimeSpan _totalTime;
private void StartTrying()
{
_started = DateTime.Now;
_totalTime = TimeSpan.Zero;
_timer = new Timer();
_timer.Interval = 1000;
_timer.Elapsed += new ElapsedEventHandler(_timer_Elapsed);
_timer.Enabled = true;
}
void _timer_Elapsed(object sender, ElapsedEventArgs e)
{
string FullPath = string.Format("{0}{1}", _watchPath, _watchFile);
FileInfo fi = new FileInfo(FullPath);
StreamReader r = null;
_totalTime = DateTime.Now - _started;
Console.WriteLine("Trying? {0}", _totalTime);
try
{
r = fi.OpenText();
r.Close();
r.Dispose();
_timer.Enabled = false;
Console.WriteLine("File is free for reading after: {0} timespan?",_totalTime);
}
catch (IOException ioex)
{
Console.WriteLine("Cannot open ? {0}", ioex.Message);
_timer.Enabled = true;
_timer.Interval += 1000;
}
finally
{
if (r != null)
{
r.Close();
r.Dispose();
}
}
}
private void OnChanged(object source, FileSystemEventArgs e)
{
// Specify what is done when a file is changed, created, or eleted.
FileSystemWatcher wo = source as FileSystemWatcher;
FileInfo fi = new FileInfo(e.FullPath);
if (wo != null)
{
switch (e.ChangeType)
{
case WatcherChangeTypes.Changed:
Console.WriteLine("File {0}{1} was changed? {2} 3}", _watchPath, _watchFile, fi.LastAccessTime ,wo.NotifyFilter);
StartTrying();
_watch.EnableRaisingEvents = false;
break;
case WatcherChangeTypes.Renamed:
Console.WriteLine("File {0}{1} was renamed? {2}",
_watchPath, _watchFile, source);
StartTrying();
_watch.EnableRaisingEvents = false;
break;
case WatcherChangeTypes.Deleted:
Console.WriteLine("File {0}{1} was deleted? {2}",
_watchPath, _watchFile, source);
StartTrying();
_watch.EnableRaisingEvents = false;
break;
case WatcherChangeTypes.Created:
Console.WriteLine("File {0}{1} was created? {2}", _watchPath, _watchFile, source);
StartTrying();
_watch.EnableRaisingEvents = false;
break;
}
}
}
public Program()
{
_watch = new FileSystemWatcher(_watchPath);
_watch.Filter = _watchFile;
_watch.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.FileName;
_watch.Changed += new FileSystemEventHandler(OnChanged);
_watch.Created += new FileSystemEventHandler(OnChanged);
_watch.Deleted += new FileSystemEventHandler(OnChanged);
_watch.EnableRaisingEvents = true;
}
static void Main(string[] args)
{
Program p = new Program();
Console.ReadLine();
}
}
}
Do you know the Debug View ?
It is a very usefull application by Microsoft. You can watch for all debug info sent to the "Debug output".
So, I was looking for debug output from some of my projects and I noticed some strange "PING/PONG" messages. The next thing I saw was some names from my Skype's contact list. Amazing ...
Shouldn't they build Skype end product in "Release" configuration and suppress all these debug output messages ?
Great surprise for all of us who in the years got used with ActionScript 2.0. First of all - we cannot apply actions directly to the movie clip (so forget about the really simple on(release){...} ).
If you click on a movie clip and you have the "Actions" window opened you will notice "Current selection cannot have actions applied to it"! At first it is irritating, but it seems to be a good point.
So how to make a "RollOver", "RollOut" etc. effects in AS3 ?
The first what we do - we have to import the Events package:
import flash.events.Event;
Now lets assume we have the movie clip on the stage and we named it "mymc". What we have to to is to add event handlers to it to listen for events:
mymc.addEventListener(MouseEvent.ROLL_OVER, RollOverHandler);
mymc.addEventListener(MouseEvent.CLICK, ClickHandler);
mymc.addEventListener(MouseEvent.MOUSE_OUT, RollOutHandler);
There is a set of predefined constants in the MouseEvent object, which we can use.
The next step is to implement the event handlers. Here is the simplest implementation of an event handler:
function RollOverHandler(me:MouseEvent):void {
me.target.alpha = 1.0
}
I would rather call the type of "me" MouseEventArgs, but it's another story. What do we have in the MouseEvent variable is the "target". The "target" is the object that has escalated the event. Now we can say to that target what to do. We can say for example: me.target.gotoAndPlay(2), or me.target.gotoAndStop(1);
OK, that's it so far. Hope to have more to write soon ;)
A long time not writing, hyh ...
Tough times ;)
Anyway, the last couple of hours I struggled with the problem: "allowFullScreen"=>"true" to enable full screen video playing in one of the latest projects I had. When I use the original generated code from the "Publish" in flash I have no problems. But when I tried with SWFObject I ran into trouble.
The full screen mode did not work... at all ... I examined the generated code (using FireBug and IE DevToolBar - developer's right hand tools) and it seemed quite OK! The parameter was there. But the flash player did not want to go full-screen.
After having a short break and coming back to the problem I noticed that I am also using the "wmode"=>"transparent" for the target movie. Commented it - just to try. Guess what - hola! The movie is again working as expected.
The problem - FullScreenMode is INCOPATABLE with TRANSPARENT movie - what the ?
Looking over this post, it seemed that: Adobe says: "...Full-screen mode was not originally supported if the wmode is opaque or transparent windowless, but it is now supported starting with the latest Flash Player 9 Update...", but I have the 9.0.45 and it still got the problem ...
A friend of mine told me "why would you need transparent along with full screen?". Well, another "great feature" of the FLASH object is that it keeps itself ALWAYS ON TOP of the page, regardless the "z-index" of your other objects. Unless it is set to "transparent".
I just noticed the BETA of Windows Live Writer and downloaded it. IT offers a WPF UI to your blogger aacount, actually it can post to bunc of BLOGS like WordPress, Blogger, Community Server and, of course Live Spaces (http://livecom.spaces.live.com/).
I find it usefule, you can try it also ;)
1. Page.xaml
1.1. Page.xaml.cs
2. Silverlight.js
3. TestPage.html
3.1. TestPage.html.js
The first one - Silverlight.js is Javascript required for the silverlight client.
Page.xaml is the Silverlight canvas itself - the project that we will work with.
TestPage.html is the HTML client page that will host our Silverlight control.
Let's go deep into it.
The Page.xaml. XAML stands for eXtensible Application Markup Language, and is pronounced "Zammel", was introduced with Microsoft.NET Framework 3.0 as the User Interface definition language for the next generation of applications. It is a normal XML file, and can be edited with any text-editing tool (we will use better than text editing). The empty project file will look like this:
That's right - we have Page.xaml.cs and this is the "code-behind" file.
Well, we can simply add one TextBlock element to this canvas:
^textblock color="#cc0000">Text="Hello World" FontSize="15" FontWeight="Bold" />
So at finally we have the following XAML code:
See you later on.
<div style="clear: both; display: block; background-color: blue;">
<div style="clear: none; float: left;">some block</div>
<div style="clear: none; float: left;">some block</div>
<div style="clear: none; float: left;">some block</div>
</div>
<div style="clear: both; display: block; background-color: blue;">
<div style="clear: none; float: left;">some block</div>
<div style="clear: none; float: left;">some block</div>
<div style="clear: none; float: left;">some block</div>
<div style="clear: both;"></div>
</div>
