Downscaling images in Flash applications

The other day I ended up in an interesting discussion regarding how to downscale a large image in a Flash based application. What we figured out was that even though the FlashPlayer can do bilinear smoothing on scaled images automatically, this will fail when the scaling factor is more than 2 times of the source image. Hence a custom method is needed.

The simple approach (scaling down less than 2 times the size)
If the loaded image is to be scaled less than 2 times (400 -> 200) simply use the build in "smoothing" property of the Bitmap Class. This can be set directly on the Bitmap itself or via different DisplayElements such as the mx:Image in Flex. If you set this property to true the Bitmap will use bilinear scaling, achieving quite good results.

Grayscale bilinear interpolation

Now, if you need to do actual resampling of an image, in other words create a new smaller image, then you need to do this manually using the "draw()" method of the BitmapData Class. Use the bitmapData.draw() method and set the "smoothing" property to true. This will give you the same result as setting the smoothing property on an Bitmap container, however it uses less memory as only the pixels for the resulting image is stored (if you remember to dispose of the original data!).

Scaling down more than 2 times
Let's say that you have an image that is quite large and you want to create a thumbnail from it. A thumbnail is often way smaller than 2 times smaller than the original image. It would be quite nice to use the above method to just scale the image, or better resample the image with the "smoothing" property set to true. However, if you try to do this the FlashPlayer will simply fail and default back to using the "nearest neighbor" algorithm. If you try it you'll soon discover that this will give you a horrible result.

What you'll need to do in this case is to do an iterative or recursive resampling of the image at hand. In layman terms this means resampling the image several times, each time no more than 2 times downscaling factor. You could call this a multi pass resampler (actually it would be a multi-multi pass resampling as the draw() method is also multi pass). You would still use the build in "BitmapData.draw()" method to accomplish this, you only divide the work into multiple jobs. This will give you a much better result. Note that in my experience, function calls in ActionsScript 3 are quite expensive, hence a iterative approach would maybe be preferable to a recursive one.

Using PixelBender
If you have decided to go for the manual approach of resampling the picture, you can use a PixelBender shader to do the job. In this case the actual resampling will be a lot faster (as the PB language is based on C), however applying the PB shader and passing the image back and forth takes time. See the speed section..

Combining the two
So, what if you have an application and you don't know in advance the downscale factor. Well, then you'll need to use both of the approaches like so:

1. Calculate the scale factor
2. If the scale factor is less than 2, set "smoothing" to true on the container or do a singe pass resampling.
3. If the scale factor is more than two, use the multi pass scaling approach. And set the "smoothing" property of the container to false (else it will be blurry).

What about speed?

Let's say that you have an image that is 1200 x 1200 pixels and you're resampling it to 200 x 200. This is a 6 times resample and will look like ass using only the "smoothing" property, but it is faster.

Let look at some hard results. In this test we see how much time is used extra compared to no "nearest neighbor" scaling (default in Bitmaps if smoothing is off).

Method	Time in ms
Smoothing property on	0 ms
Multi pass resample	9 ms
Multi pass PixelBender	32 ms

Using the smoothing property takes no extra time, because it fails and defaults back to nearest neighbor. The other two are both pretty fast, however even though the PB algorithm is way faster than the others, we see that the applying of the shader eats a lot of time, thereby making the multi pass resampler method the way to go on a large rescaling operation.

Screencast # 2 - Scrolling HBox without scrollbars (Flex 3)

In this screencast we'll be looking at how to scroll an HBox in Flex 3, without having the scrollbars present. I'll also show you how to get a nice animation effect when doing the scrolling.
Source code below the video.



Download source

Screencast # 1 - Filtering ArrayCollections

In this first screencast I take look at loading XML data into an ArrayCollection, avoiding some common pitfalls and then doing live filtering on the data shown in an DataGrid.
Note that this is my first sceencast and my native language is not english, so this is just me getting my feet wet so to speak.
Source code below the video.



Download source

Quicktip: Searching an ArrayCollection fast and easy

Let's say your Flex project needs to maintain a large amount of data, like all the userdata from you big ass site. You can use several datastructures in Flex to store complex userdata, from XML to linked lists. However, perhaps the most useful and easiest structure is the Flex native ArrayCollection. This structure gives you a lot for free, for instance.. searching for stuff.

Let's say you have all of your users in "usersCollection" which is an ArrayCollection. A user object may contain things like "firstNamename","lastName", "email", "address" and so on. Now, what if you need to find a specific user like "John Rambo". Then you'll need to search both the first and last name fields. You could of course traverse the entire collection using a loop, checking every iteration. Hang on, there is a faster way using a cursor (it's kinda lika a struct pointer if you're a C person).

First you need to sort the collection

var mySort:Sort = new Sort();

mySort.fields = [new SortField('lastname')];

usersCollection.sort = mySort;

usersCollection.refresh();

Then we create the cursor, pointing to the first item in the collection at init.

var cursor:iViewCursor = usersCollection.createCursor;

Now, we need to search both first and last name, so we create a searchobject.

var sObj:Object = {firstname:"John",lastname:"Rambo"};

We are now ready to do the actual search by using the findAny method of the cursor.

if(cursor.findAny(sObj)){

//We found him

}else{

//John Rambo is lost again

}

In order to get to the user we found we utilize the current property of the cursor.

var foundUser:Object = cursor.current;

Do keep in mind that the cursor is now pointing to the user found, so if we want the next user in the list we can simply go "cursor.moveNext()"

Why should you use this method? It's more flexible, it's faster and uses less resources than the traditional iterative loop approach.

Adobe Flash becoming an "Ad tool" full time?

As reported by Ryan Stewart, Adobe is altering the name of the next version of Flex Builder(code name «Gumbo») to «Flash Builder». Seeing this in correlation with the new «Flash Catalyst», I can't help wonder where this leaves the Flash authoring tool.

Adobe Flash has for some time been struggling with a bad rep for being the tool "which is used for creating annoying banner ads". This view does hold water in many cases, adding insult to injury the fact that many Flash sites are poorly built and tend to leave users with a crashing web-browser. It seems like Adobe are segmenting out Flash as a cowboy tool, fronting Flash Catalyst and Flash Builder as the tools for creating "real" RIA applications and "robust" rich interactive web sites. On the other hand this may not be their intention at all, however I do believe it will be the result following this strategy.

I base this assumption on two things in large. One is the fact that the Flash authoring tool becomes more or less redundant with the introduction of Flash Catalyst, seeing it from a designer standpoint. As for developers, the Flash authoring tool has never been the first choice to put it in subtle terms. The second argument bases itself in the evolutionary direction in which rich internet applications are heading. Current trends shows that rich web-sites and RIA applications grow ever more complex, at the same time users are demanding a more friendly UI, conformity in the experience and short load times. Flex and Catalyst performs advanced tasks better..

Adobe Flash authoring is a tool for creating interactive animations, but has been used for everything else in the past years. I do not think the Flash authoring tool will die, but it's role will diminish, leaving Flash Catalyst as the main designer tool for rich content and Flash Builder as the developers choice.

Generating a thumbnail and saving it to disk

This quicktip will show you how to load an image from wherever and how to generate a thumbnail from that image and save it to the disk. In fact, you can use this method to generate screen dumps from any visual component from within Flex.

There are some considerations, first the actual scaling of an image can be done in several ways, the most used trough a scale Matrix. However, in this example I'll be utilizing the Flex library in order to scale the image using smoothing, thereby accomplishing a better result than with simply a Matrix.

Let's say I have loaded an image into a Flex Image component(myImg) and wish to generate a widescreen(16/9) thumbnail for it.

Load the image

myImg.source = "http://graphics8.nytimes.com/images/2006/12/28/business/28dog.xlarge1.jpg";

When the image is finished loading(Event.COMPLETE) we generate the thumb at a size of 200x112

var orgBitmap:BitmapData = new BitmapData(myImg.contentWidth,myImg.contentHeight);

var thumb:BitmapData = new BitmapData(200,112);

orgBitmap.draw(myImg);//Get the orginal image

var scaleImg:Image = new Image();

scaleImg.load(new Bitmap(orgBitmap,"auto",true));//Set smoothing to true

scaleImg.content.width = 200;

scaleImg.content.height = 112;

thumb.draw(scaleImg);//Here we have the scaled image data

Now we have scaled the image using the Image component in Flex which actually does smoothing on the image when scaled. Now we need to convert the bitmapdata into valid JPEG format.

var myjpeg:JPEGEncoder = new JPEGEncoder(80);//Set quality to 80

var thumbdata:ByteArray = myjepeg.encode(thumb);

Now we need to save it to disk(to desktop)

var fl:File = File.desktopDirectory.resolvePath("myimage.jpg");

var fls:FileStream = new FileStream();

fls.open(fl,FileMode.WRITE);

fls.writeBytes(thumbdata);

fls.close();

There you have a smoothed thumbnail waiting for you on the desktop.

Coding the DocumentClass in Flex Builder

Since the dawn of Flash CS3, developers have been given the grace of the DocumentClass(explained). The DocumentClass effectively replaces the «lets write all our code on frames» approach which, frankly was more of a hackers approach to coding within the Flash IDE. The main advantage of the DocumentClass is it's life outside of the Flash IDE, it's a separate .as file.
However, there are some things to consider when jumping in to a project using the DocumentClass. One of these considerations must be, that the built in code editor in the Flash IDE, well..it's not the best environment in to be writing code. Alternative ActionScript editors exists, like TextMate(OS X) or FlashDevelop(Win). But the far superior cross platform alternative in my opinion, is Flex Builder. FB is great for many reasons, mainly because it utilizes the power of the Eclipse platform, an environment well known to Java and CF developers.
In short, Flex Builder is a powerful coding environment. But there are some considerations to take when working with a DocumentClass to be exported trough Flash.. one of them is that Flex cannot see your instance names(objects on the stage) and give you code completion on them..in fact, it will complain and claim it's an error. This can be fixed in a jiffy..

• In Flash, open the project settings.


• Click the «Settings» button to the right of the «script» drop down


• Unckeck «Automatically declare stage instances»

Now Flex will let you declare your own instances from within the DocumentClass, like they where properties of the DocumentClass itself(which they are). However, you will need to define the instances as «public» properties/variables, so the Flash compiler can see them. Okay, time for an example.

Let's say I have a blue movieclip on the stage with an instance name of "cheeseball". Here is how I access this movieclip from within my DocumentClass.

package{
import flash.display.Sprite;
import flash.display.MovieClip;

public class MainClass extends Sprite{

public var cheeseball:MovieClip;//<-----

public function MainClass(){
trace("The name of the clip is "+cheeseball.name);
}
}
}


As you can see, the «cheeseball» movieclip on the stage is now accessed by defining it as a public property, making it visible to both Flash and Flex, giving you code completion, error checking and all kinds of wonderful things. Do this and your day will be shinier!

QuickTip: MOUSE_OVER vs ROLL_OVER

So, what is the difference between the MouseEvents, MOUSE_OVER and ROLL_OVER? The short answer is that MOUSE_OUT triggers when mousing over a child of the listener object, ROLL_OUT only triggers when leaving the listener objects bounding box. Consider the following code:

var sp:Sprite = new Sprite();
sp.graphics.beginFill(0xFF0000);
sp.graphics.drawRect(0,0,200,200);
sp.graphics.endFill();

var inner:Sprite = new Sprite();
inner.graphics.beginFill(0x00FF00);
inner.graphics.drawRect(0,0,50,50);
inner.x = 75;
inner.y = 75;
sp.addChild(inner);
addChild(sp);

sp.addEventListener(MouseEvent.MOUSE_OVER, report);
sp.addEventListener(MouseEvent.MOUSE_OUT, report);
sp.addEventListener(MouseEvent.ROLL_OVER, report);
sp.addEventListener(MouseEvent.ROLL_OUT, report);

function report(evt:MouseEvent):void{
trace(evt.type);
}


The sprite "inner" is a child of "sp". When rolling over "sp", both the MOUSE_OVER and the ROLL_OVER will trigger. However, when rolling over "inner", MOUSE_OUT will trigger, succeeded by MOUSE_OVER again. ROLL_OUT will not trigger when rolling over "inner". Test it out and it will all become clear!

Flash and Flex, bridging the gap

This post is a followup of my previous post regarding Flex and Flash. Although this is a followup, the focus has changed in parallel with the internal functionality and focus of Flex Builder and Flash. Back in august 2007, Flex and Flash existed in two different worlds, this is still the case, however now there are bridges between the two worlds.
As a developer, Flex (Builder/Eclipse) is a tool far superior to the Flash editor from my point of view. But when it comes to flexible design, Flex Builder still brings little to none to the table. And as far as my understanding of Adobe strategy goes, this is done intentionally. Flash should be an animation and design tool, whereas Flex Builder focuses on the developers. Adobe does intentionally try to separate the two..smart move!

What Adobe has done to amend for the intentional wedge driven in between Flash and Flex, is to create ways for developers to remain in Flex Builder while "importing" design created from within Flash. This is done trough bridging tools like the Flex component kit and the Flex component templates. This enables designers to stay in Flash(or any other CS product), using tools familiar to them, creating the design to be implemented via Flex. This approach to creating rich interactive content is much better than the old one involving big ass .fla files and no code help what so ever(almost). The snag is, when Adobe decides to change the approach of the designer developer workflow, change is also forced upon a bunch of Adobe/Macromedia users. To change their entire workflow may seem as a daunting task for many designers and developers. However this change is a change for the better, it's just a matter of dropping the conservative mindset.

As of now, the Flex Component kit is the best way to prepare for the real transition in my mind. You can of course go ahed and do lots of CSS styling in Flex builder, not a good idea unless there's only one person working on the project, as Flex builder is needed to see the design as a whole. The real transition is of course the introduction of FXG, the new interchange format making it possible to do full designs in CS products, add interactivity in the upcoming Flash Catalyst, then do the development in Flex Builder. In this way the different tools will keep their strengths and not worry about unrelated functionality, Flash becomes a design/animation tool and Flex a development tool exclusively. Like the current workflow of movie editing suites like FinalCut Studio, you will be able to use the FXG format to pass files back an forth between the designer and the developer, maintaining the focus, never worrying about stuff outside your field of your expertise. This is a long awaited and a big step for Adobe. Along with AIR, this move will in my opinion reinforce the FlashPlayer status as the goto(pun intended) technology for cross-platform interactive content and development.

Optimizing Flash performance

Introduction
As Flash designers and developers get a hold of even more advanced tools and scripting languages, the need for performance considerations within flash projects become ever so apparent. Lately I've seen people complaining about flash apps that bug down their CPU and memory when visiting sites containing flash media. I see this problem as two parallel problems.

1. Sites relying on advertisement (often flash banners) try to cram a gazillion banners and ads into every nook and cranny available on their site. This is simply greed, a condition for which I can offer no cure.

2. The Flash editor has been an animation tool for quite some time. As a result, things common to other development platforms, like performance optimization, has not been and still isn't a priority in most Flash projects. However, this has started to change, something I will try to contribute into trough this post. So, let's get it on...

General optimization

• WMODE. First of all, if you don't have an amazingly good reason for using it, do not use wmode! In short, wmode is used to make the background of the swf file become transparent. Wmode will render the swf unreadable to screen readers, force the swf to run when on a hidden tab and make the content of your swf to render much more slowly than in normal mode. Read more about this (Tinic Uro).
• Vectors and bitmaps. If you are animating a complex static vector, that is a vector that does not change it's content, e.g. a rectangle, then use the «cacheAsBitmap» property or simply use a bitmap instead. Let's say you are tweening a vector shape across the stage. The Flash-Player(FP) will have to recalculate all the vectors in the shape for every frame. For complex shapes, this will take up a lot of CPU cycles. With «cacheAsBitmap» turned on, the FP will calculate the vectors once, create a bitmap and re-render the bitmap instead. But do note this!! For vector shapes that are changing, not static, like lines that are constantly moving inside of a movieclip, if so..do not use «cacheAsBitmap». The reason for this is that FP now have to re-render both the changed vector shape AND create a new bitmap for every frame. So «cacheAsBitmap» is only good for static vector shapes.
• MovieClips. Not every shape needs to be a movieclip. If you need frames, you do need a movieclip, but if not, use a sprite(in ActionScript) or a Shape in the editor. Shapes does not have mouse and keyboard events and leave a much smaller memory imprint than MovieClips.
• Alpha. Crop your artwork instead of using big ass areas of alpha channel. The FP do render the alpha channel areas even if you cannot see them.
• Alpha video. A flv video encoded with an alpha channel using the On2Vp6 codec is about 4 times more complex to decode than one without the alpha channel. FP do not support alpha using H.264. Read more.
  

ActionScript optimization
This section is on ActionScript 3 only and can get somewhat technical.

• Strong typing. By typing your variables and functions the performance gain is substantial, in general 10 times faster. In AS3 the byte-code produced by the compiler actually changes based on typing. Also, strong typing makes it much easier to debug your app as the compiler will give more accurate debugger results. More on this.
• Numbers. Even though strong typing is quite easy to do, it's not always clear as to what type is the most efficient one. For numbers the «int» type is the speediest of the bunch, however «Number» does not lag far behind. However, it's smart to use «int» for iterations and situations where you are working on integers only as this will make debugging more effective. But do avoid «uint», unless you really need it. The «uint» type is a turtle compared to «Number» and «int». More on this here.
• Local variables. If you are going to access a variable more than two times, store it in a local variable first. Local variables are faster because they do not require a lookup into objects. E.g:
  var fName:String = person.firstname;
if(fName == "jim" || fName == "jill")
  is faster than
  if(person.firstname == "jim" || person.firstname == "jill")
  
• Data binding. Binding in Flex is a valuable and efficient way of handling data, however bindings do take up memory and they are general type. In other words bindings don't know what part of the data that will change, therefore all the data needs to be watched. If you are going the access the data only once or twice, it's better to do an manual assignment in code.
  

That's all folks.. be sure to check out the links as well.

Quicktip:Event handling

I've been seeing a lot of this(image) following the launch of ActionScript 3. For me, having the debugger version of Flash Player installed it manifests as an error dialog box telling me that loading did never complete. If you don't have the debugger version of the Flash Player this will only result in the application stopping without any message.

What puzzles me a bit is how often I see this error and the share number of professionally build Flash sites where this occurs. Now, this is completely avoidable. It's just a question of handling your events.

Basically there are 6 event you handle when loading stuff over http(s). You don't actually need to do anything upon the occurrence of an event(however it is smart to do so), you simply need to handle it in order to avoid FP throwing an error shown above. Do this and your app will feel so much more well made to the end user.

Sound visualizer using SoundMixer

This time you'll get a complete class drawing a visualizer graph for the sound playing(or, actually the bytes in the sound). You can show the graph as a raw sound wave or as grouped by frequency(where a fourier transformation has been applied). This is just a simple demo, but you can separate out different frequencies to get some really cool effects. Play around with it and see what you get. PS: You do need to provide your own sound ;-)

package {
import flash.display.GradientType;
import flash.display.MovieClip;
import flash.events.Event;
import flash.media.Sound;
import flash.media.SoundMixer;
import flash.net.URLRequest;
import flash.utils.ByteArray;

[SWF(width='512',height='300',backgroundColor='0x000000',frameRate='30')]

public class SpectrumTest extends MovieClip{

private var _eql:Boolean = true;//Show as grouped by frequenzy
private var _completeGraph:Boolean = false;//Complete graph fill
private var _spikeAmp:uint = 100;//Amplifier for graph peaks
private var _graphFill:Array = new Array(0xFF0000, 0xFFDF21);//Colorfill for graph

private var _soundBytes:ByteArray;

public function SpectrumTest(){
var s:Sound = new Sound(new URLRequest("yoursoundfile.mp3"));
s.play(0, 3);
this._soundBytes = new ByteArray();
this.addEventListener(Event.ENTER_FRAME, drawGraphHandler);
}

private function drawGraphHandler(evt:Event):void{
graphics.clear();
graphics.clear();
graphics.beginGradientFill(GradientType.LINEAR, this._graphFill, [1,1],[0,255]);
graphics.moveTo(-1, 150);

//Get the bytes
SoundMixer.computeSpectrum(this._soundBytes,this._eql);

for(var i:uint=0; i<(stage.stageWidth * .5); i++){ 

var num:Number = -this._soundBytes.readFloat() * this._spikeAmp + stage.stageHeight * .5; 

graphics.lineTo(i*2, num); 

} 

if(this._completeGraph){ 

graphics.lineTo(512, 300); 

graphics.lineTo(0, 300); 

graphics.lineTo(-1, 150); 

} 

} 

} 

}

Briefly: SWF and FLV formats opens up

Adobe has announced the Open Screen Project which will be one big step in the openness direction of the SWF and FLV file formats. In practice this means that you can build your own Flash player, or an FLV flayer. So, iPhone Flash player anyone?

Ryan Stewart has written a more in depth explanation on what this announcement means.

Quicktip: Skewing an image using ActionScript 3

So you want to skew an image, or a MovieClip or perhaps any other DisplayObject. You could do this manually in the Flash editor of course, but what if you need to do it dynamically, or in Flex.. Here is how.

//If you prefer to draw one visually, skip down
/*var target:MovieClip = new MovieClip();
target.graphics.beginFill(0x000000);
target.graphics.lineTo(100,0);
target.graphics.lineTo(100,100);
target.graphics.lineTo(0,100);
target.graphics.lineTo(0,0);
target.graphics.endFill();
//center it on the stage
target.x = stage.stageWidth *.5-50;
target.y = stage.stageHeight *.5-50;
//add the mc to the stage
addChild(target);*/
/*****/
//Set the skewing angle
var degX:Number = 15;
var degY:Number = 15;



//Get the transform matrix for the object to skew
var m:Matrix = target.transform.matrix;
m.b = Math.tan(degY *(Math.PI/180));
m.c = Math.tan(degX +(Math.PI/180));



//Apply the matrix to the transform object
var t:Transform = new Transform(target);
t.matrix = m;



//Apply the skew
target.transform = t;


Looks like a lot of code, but remember that the first 10 lines of code(not counting comments) are just to draw a MovieClip onto the center of the stage. The actual skew is achieved in just 8 lines of code.

When applying this to a movieclip it will look something like the image in the top of the post.

Quicktip: Ducking local security in FlashPlayer

Ever needed to test a swf containing networking residing on your local hard drive? Sure, you can choose to export for only network from the publish menu in Adobe Flash. But what if you need to load some movies locally and then make a connection to your remote server at the same time? Well, out of the box, you can't. But wait, there is a simple solution to allow flash player to do both when playing locally. Only tree easy steps..

1. Open your favorite texteditor(BBEdit, Coda, Textmate..) and create a new textfile. At the first line, type the path of the folder where your project resides, e.g./Users/username/myproject

2. Save the file with a .cfg extension to: /Library/Application Support/Macromedia/FlashPlayerTrust/

3. There is no step 3

Note: You can do this for your entire user account, but I strongly recommend not to do so, cos this will allow not trusted files to run both networking and local command on your computer if you download them. You don't want that.

Quicktip: Avoiding popup blockers in AS3

Adobe/Macromedia did change a couple of things as the release of ActionScript 3 became a reality.

For once, it's not as easy as it once was to open an external browser window. There are many solutions to this floating around, many of them involve using the "wmode" parameter in the HTML embed code for the swf. In my opinion this is not a good idea cos it will bring a bunch of new problems with it, like handling textfields and so on. I've written a possible solution for this problem which works using standard embed code in the HTML. It simply tries to use the ExternalInterface, if that fails, it falls back to navigateToURL. Hey, remember to import ExternalInterface, URLRequest and NavigateToURL in order for it to compile properly.

private function openURL(url:String, window:String = "_blank") : void {
var WINDOW_OPEN_FUNCTION:String = "window.open";
if(!ExternalInterface.call(WINDOW_OPEN_FUNCTION, url, window)){
navigateToURL(new URLRequest(url),window);
}
}


Quicktip: Changing the font in Flex 3 editor

Have you ever wanted to turn enlarge that font size in the code editor in Flex builder 2/3, but have thrown in the towel after several hours poking around to find this setting? Since Flex is build on Eclipse, this part of the editor is actually identical to the Eclipse interface. To change the font, choose Window -> Preferences -> General -> Appearance -> Colors and fonts.

In that small list of folders, choose Basic -> Text font and hit the change button. Tada! Kinda a lot of effort, but it will be worth it in assisting your eyes from squinting all day long.

And hey, while we're at it. If you are starting to type something in the code editor, but can't quite remember the whole variable name or whatever... hit ctrl+ spacebar. Flex will then make a suggestion. This is a very handy feature as well..