Tuesday, January 11, 2011

Introducing Google Instant



Google Instant is a new search enhancement that shows results as you type. We are pushing the limits of our technology and infrastructure to help you get better search results, faster. Our key technical insight was that people type slowly, but read quickly, typically taking 300 milliseconds between keystrokes, but only 30 milliseconds (a tenth of the time!) to glance at another part of the page. This means that you can scan a results page while you type.
The most obvious change is that you get to the right content much faster than before because you don’t have to finish typing your full search term, or even press “search.” Another shift is that seeing results as you type helps you formulate a better search term by providing instant feedback. You can now adapt your search on the fly until the results match exactly what you want. In time, we may wonder how search ever worked in any other way.

Benefits

Faster Searches: By predicting your search and showing results before you finish typing, Google Instant can save 2-5 seconds per search.
Smarter Predictions: Even when you don’t know exactly what you’re looking for, predictions help guide your search. The top prediction is shown in grey text directly in the search box, so you can stop typing as soon as you see what you need.
Instant Results: Start typing and results appear right before your eyes. Until now, you had to type a full search term, hit return, and hope for the right results. Now results appear instantly as you type, helping you see where you’re headed, every step of the way

Saturday, January 1, 2011

NodeXL: Network Overview, Discovery and Exploration in Excel

NodeXL is a powerful and easy-to-use interactive network visualisation and analysis tool that leverages the widely available MS Excel application as the platform for representing generic graph data, performing advanced network analysis and visual exploration of networks. The tool supports multiple social network data providers that import graph data (nodes and edge lists) into the Excel spreadsheet.

The tool includes an Excel template for easy manipulation of graph data:

Sample networks generated with NodeXL:

Project contributers:

The NodeXL Template in Action

This is what the NodeXL template looks like. In this example, a simple two-column edge list was entered into the Edges worksheet, and the Show Graph button was clicked to display the network graph in the graph pane on the right.

NoAttributes.png

The two-column edge list is all that’s required, but you can extensively customize the graph’s appearance by filling in a variety of optional edge and vertex columns. Here is the same graph after color, shape, size, image, opacity, and other columns were filled in.

WithAttributes.png

In the next example, NodeXL has imported and displayed a graph of connections among people who follow, reply or mention one another in Twitter:

20100905-NodeXL-Twitter-GOP-500px.jpg


Some NodeXL Features

IntroTutorial-Layout-v2_GraphImage-2.png

NodeXL Graph Gallery

Here are some graphs that were created with NodeXL. Additional images can be found at http://www.flickr.com/photos/marc_smith/sets/72157622437066929/.

GraphGallery01.gifGraphGallery03.jpg
John CrowleyCody Dunne

GraphGallery04.gifGraphGallery02.gif
Marc SmithPierre de Vries

GraphGallery06.gifGraphGallery05.jpg
Eduarda Mendes RodriguesTony Capone

GraphGallery08.gifGraphGallery07.gif
Tony CaponeEduarda Mendes Rodrigues

Project Trident: A Scientific Workflow Workbench


Project Trident imagery
With Project Trident, you can author workflows visually by using a catalog of existing activities and complete workflows. The workflow workbench provides a tiered library that hides the complexity of different workflow activities and services for ease of use.

Version 1.2 Now Available

Project Trident is available under the Apache 2.0 open source license.

About Project Trident

Built on the Windows Workflow Foundation, this scientific workflow workbench allows users to:

  • Automate analysis and then visualize and explore data
  • Compose, run, and catalog experiments as workflows
  • Capture provenance for each experiment
  • Create a domain-specific workflow library to extend the functionality of the workflow workbench
  • Use existing services, such as provenance and fault tolerance, or add new services
  • Schedule workflows over HPC clusters or cloud computing resources

Current Status: Microsoft Research is working in partnership with oceanographers involved in the NEPTUNE project at the University of Washington and the Monterey Bay Aquarium, to use Project Trident as a scientific workflow workbench. A workflow workbench prototype has been developed for evaluation and is in active use, while an open source version is being implemented for public release.

Learn More

Project Trident in Action—Videos

Papers, Presentations, and Articles

Partner Highlight

  • myExperimentmyExperiment team
    University of Manchester and Southampton University. myExperiment makes it easy to find, use and share scientific workflows and other files, and to build communities. Project Trident uses myExperiment as the community site for sharing workflows, along with provenance traces.

Background: Project Trident

Project Trident: A Scientific Workflow Workbench began as a collaborative scientific and engineering partnership among the University of Washington, the Monterey Bay Aquarium, and Microsoft External Research that is intended to provide Project Neptune with a scientific-workflow workbench for oceanography. Later, Project Trident was deployed at Johns Hopkins University for use in the Pan-STARRS astronomy project.

An increasing number of tools and databases in the sciences are available as Web services. As a result, researchers face not only a data deluge, but also a service deluge, and need a tool to organize, curate, and search for services of value to their research. Project Trident provides a registry that enables the scientist to include services from his or her particular domain. The registry enables a researcher to search on tags, keywords, and annotations to determine which services and workflows­—and even which data sets—are available. Other features of the registry include:

  • Semantic tagging to enable a researcher to find a service based on what it does, or is meant to do, and what it consumes as inputs and produces as outputs
  • Annotations that allow a researcher to understand how to operate the registry and configure it correctly; the registry records when and by whom a service was created, records its version history, and tracks its version

The Project Trident registry service includes a harvester that automatically extracts Web Services Descriptive Language (WSDL) for a service, to allow scientists to use any service as it was presented. Users simply provide the Uniform Resource Identifier (URI) of the service, and the harvester extracts the WSDL and creates an entry in the registry for the service. Curation tools are available to review and semantically describe the service before moving it to the public area of the registry.

Because the end users for Project Trident are scientists rather than seasoned programmers, the workflow workbench also offers a graphical interface that enables the user to visually compose, launch, monitor, and administer workflows.

Project Trident: Logical Architecture

Tridente Logical Architecture

Saturday, December 25, 2010

Fed up with blue screen error at least you can change color

Seeing a bluescreen that’s not blue is disconcerting, even for me, and based on the reaction of the TechEd audiences, I bet you’ll have fun generating ones of a color you pick and showing them off to your techy friends. I first saw Dan Pearson do this in a crash dump troubleshooting talk he delivered with Dave Solomon a couple of years ago and now close my Case of the Unexplained presentations with a bluescreen of the color the audience choses (you can hear the audience’s response at the end of this recording, for example). Note that the steps I’m gong to share for changing the color of the bluescreen are manual and only survive a boot session, so are suitable for demonstrations, not for general bluescreen customization. Be sure to check out the special holiday bluescreen I’ve prepared for you at the end of the post.

Preparing the System

Because you’re going to modify kernel code, the first step is to enable the ability to edit kernel code in memory if it’s not already enabled. Windows systems with less than 2 GB of RAM uses 4KB pages to store kernel code, so can protect pages with the protection most suitable for the contents they contain. For instance, kernel data pages should allow both read and write access while kernel code should only allow read and execute access. As an optimization that helps improve the speed of virtual address translations, Windows uses large pages (4 MB on x86 and x64) on larger systems. That means that if there’s both code and data stored in a page, the page must allow read, write and execute accesses, so to ensure that you can edit a page, you have to encourage Windows to use large pages. If your system is Windows XP or Server 2003 and has less than 256 MB, or is Windows Vista or higher and has 2 GB or less of RAM, create a REG_DWORD value called LargePageMinimum that’s set to 1 under HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management:

image

So that you don’t have to rush to show off your handiwork before Windows automatically reboots after the crash, change the auto-reboot setting. On Windows XP and Server 2003, right-click on My Computer, select the Advanced Tab, and press the Settings button in the “Startup and Recovery” section. On Windows Vista and higher, right-click on Computer in the Start Menu, select properties to open the Properties dialog, click Advanced System Settings, select the Advanced tab and press the Settings button in the “Startup and Recovery” section. Finally, uncheck the “Automatically restart” checkbox:

SNAGHTML5fc0cb41_thumb2

If you’re running 64-bit Windows Vista or higher, you need to boot the system in Debug mode so that you can run the kernel debugger in “local” mode. You can do that either by selecting F8 during the system boot and choosing the Debug boot or by checking the Debug checkbox in the System Configuration (Msconfig) utility:

image_thumb31

Next, reboot the system and start the debugger with administrator rights (if UAC is on, run it as administrator). Point the debugger at the Microsoft symbol server by opening the Symbol Search Path dialog under the File menu and enter this string: srv*c:\symbols*http://msdl.microsoft.com/download/symbols (replace c:\symbols with whatever local directory in which you want the debugger to store cached symbols). Next, open the Kernel Debugging dialog from the File menu, click the Local page, and press OK:

image_thumb33

The subsequent steps vary depending on whether you’re running 32-bit or 64-bit Windows and whether it’s Windows Vista or newer.

32-bit Windows XP and Windows Server 2003

The function that displays the bluescreen on these operating systems is KeBugCheck2. You’re looking for the place where the function passes the color value to the function that fills the screen background, InbvSolidColorFill. Enter the command “u kebugcheck2” to list the start of the function, then enter the “u” command to dump additional pages of the function’s code until you see the reference to InbvSolidColorFill (after entering “u” once, you can just press enter to repeat the command). You’ll need to dump 30-40 pages before you come across the one with the call:

image

Preceding the call, you’ll see an instruction that has the number 4 as its argument (“push 4”), as you can see above. Copy the code address of that instruction by selecting it from the address column on the left and typing Ctrl+C. Then in the debugger command window, type “eb “, then Ctrl+V to paste the address, then “+1”, then enter. The debugger will go into memory editing mode, starting with the address of the color value. Now you can choose the color you want. 1 is red, 2 is green, and you can experiment if you want a different color. Simply enter the number and press enter twice to commit it and exit editing mode. Here’s what the screen should look like after you’re done:

image_thumb38

64-bit Versions of Windows and 32-bit Windows Vista and Higher

On these versions of Windows, the core bluescreen drawing function is KiDisplayBlueScreen. Type “u kidisplaybluescreen” and then continue entering “u” commands to dump pages of the function until you see the call to InbvSolidColorFill. On 32-bit versions of Windows, continue by following the instructions given in the Windows XP/Server 2003 section to find and edit the color value. On 64-bit versions of these operating systems, the instruction preceding the call to InvbSolidColorFill is the one that passes the color, so copy its address (the number in the left column) and enter this command to edit it: “eb

+4”. The debugger will go into memory editing mode and you can change the value (e.g. 1 for red, 2 for green):

image_thumb42

Viewing the Result

You’re now ready to crash the system. If you’re running 64-bit Windows, you might get a crash without doing anything additionally. That’s because Kernel Patch Protection will notice the modification and crash the system as a deterrent to ISVs that might consider modifying the kernel’s code to change its behavior. There might be a delay of up to several minutes before that happens, though. To generate a crash on demand, run the Notmyfault tool (you can download it from the Windows Internals book page) and press the “Do Bug” button (to avoid data loss, make sure you’ve saved any work and closed all other applications):

image_thumb45

You’ll now get a bluescreen in the color you picked, in this case the red screen of death:

image_thumb47

The Holiday Bluescreen

In the spirit of the holiday season, I took things one step further to generate a holiday-themed bluescreen: not only did I modify the background color, but the text color as well. To do this on 64-bit versions of Windows Vista or higher, note the call to InvbSetTextColor immediately following the one to InvbSolidColorFill and the address of the instruction that passes the text color to the function, “move ecx, 0Fh”:

image

The 0Fh parameter represents white, but you can change it using the same editing technique. Use the “eb” command, passing the address of the instruction plus 1. Here I set the color to red (which is a value of 1):

image

And here’s the festive bluescreen I produced:

image

Happy holidays! And remember, if you have any troubleshooting cases you want to share, please send me screenshots (.PNG preferred) and log files.

Read more at Mark's Blog ...

Few good memory management tools, you really need!

ProcDump v3.0
This update to ProcDump, a flexible command-line utility for capturing process dumps based on time, CPU, memory, or performance counter thresholds, adds a new dump type, Minidump Plus, that uses heuristics to create the equivalent of full dumps for very large processes, but with large data sections and caches omitted. This has been shown to reduce the size of Exchange process (store.exe) dumps by 50-90%. Thanks go to Andrew Richards from Exchange support at Microsoft for this contribution.

Process Explorer v14
This major update to Process Explorer adds a slew of enhancements and new functionality including network and disk monitoring, an improved multi-tab system information dialog, additional memory statistics, a new column that shows aggregate CPU usage for a tree of processes, improved DLL scanning performance and accuracy, command-lines in process tree tooltips, support for more than 64 CPU systems, and more.

VMMap v3.0
This major update to VMMap, an advanced process memory-analysis utility, now shows locked virtual memory, records multiple memory snapshots, and has a timeline view that enables you to load older snapshots into the main view and compare any two snapshots from a given execution. In addition, you can now launch processes from VMMap so that VMMap saves periodic snapshots and records a trace of the process’s virtual memory and heap operations. New trace, heap and call tree views list the recorded operations, show heap block sub-allocations including stack traces, and let you see a complete view of all the places in the process that invoked the traced memory operations.

LiveKd v5.0
LiveKd, a tool that enables live kernel debugging of Windows systems, can now debug and generate kernel dump files of Hyper-V Windows virtual machines from the parent partition without having to boot the target virtual machine in debug mode. See Mark’s most recent blog post, LiveKd for Virtual Machine Debugging, for more information.

Sysinternals Suit
The Sysinternals Troubleshooting Utilities have been rolled up into a single Suite of tools. This file contains the individual troubleshooting tools and help files. It does not contain non-troubleshooting tools like the BSOD Screen Saver or NotMyFault.


Mysteries of Windows Memory Management, Part 1
Mysteries of Windows Memory Management, Part 2
If you want to know the difference between System Committed memory and Process Committed memory, wondered what all those memory numbers shown by Task Manager really mean, or want to gain insight into the memory-related impact of a process, then this talk is for you. Watch Mark in this on-demand webcast from the 2010 Professional Developer’s Conference.