Classic post about Empirical Bayesian application in MEG source reconstruction.

Dear Yuri,

Yury Petrov wrote:
> Hi Will,
> 
> I attached the paper. 

Thx, its a top paper.

My concern is that the EM algorithm cannot be
> used to estimate two parameters when one of them is used to define a
> prior for the other. 

It can.

One parameter defining a prior over another results in a hierarchical 
model. Bayesian estimation of linear Gaussian hierarchical models was 
solved in the 70's by the stats community. More recently the machine 
learning community have been using various approximate inference 
algorithms for hierarchical nonlinear/nonGaussian models. See 
Jordan/Bishop/Ghahramani etc.

Irrespectively of how the MSP algorithm has been
> derived, the ReML learning part explicitly described in the Appendix
> of the Phillips et al 2002 paper is violating the Bayes rule. It
> first calculates the source covariance matrix given the solution of
> the previous iteration, then uses its scale (trace) to rescale the
> original source covariance, etc. Yes, it uses the 'lost degrees of
> freedom' trick 

This isn't a trick. It falls naturally out of the mathematics.

to prevent a nonsensically localized solution, but
> this trick does not address the main problem. The algorithm still
> changes the prior based on posterior, then posterior based on the new
> prior, etc. iteratively.
> 

All of what i've said corresponds to the framework of Empirical Bayes - 
where you estimate the parameters of priors from data.

Pure Bayesians do not allow this. They see it, as you say, as a 
violation of what a prior is.

But then pure Bayesians have'nt solved many interesting problems. The 
Empirical Bayesian claims to know only the form of prior densities. Not 
their parameters.

Best,

Will.

> 
> 
> ------------------------------------------------------------------------
> 
> 
> 
> 
> On Sep 22, 2010, at Sep 22, 2010 | 1:14 PM, Will Penny wrote:
> 
>> Dear Yury,
>> 
>>>> ---------------------------------- Dear All,
>>>> 
>>>> I have a conceptual concern regarding the MSP algorithm used by
>>>>  SPM8 to localize sources of EEG/MEG activity. The algorithm is
>>>>  based, in part, on EM iterative scheme used to estimate source
>>>>  priors (source covariance matrix) from the measurements. The
>>>> way this scheme is described in the Phillips et al. 2002 paper,
>>>> it works as an iterative Bayesian estimator: first it estimates
>>>> the sources, then calculates the resulting source covariance
>>>> from the estimate, next it (effectively) uses it as the new
>>>> prior for the sources, estimates the sources again, etc.
>>>> However, applying Bayesian learning iteratively is a common
>>>> pitfall and should not be used, because each such iteration
>>>> amounts to introducing new fictitious data. I attached a nice
>>>> introductory paper illustrating the pitfall on page 1426.
>> 
>> I don't believe that this is a pitfall.
>> 
>> The parameters of the prior (specifically the variance components)
>> are estimated iteratively along with the variance components of the
>> likelihood.
>> 
>> Importantly, each is estimated using degrees of freedom which are 
>> effectively partitioned into those used to estimate prior variance
>> and those used to estimate noise variance. This is a standard
>> Empirical Bayesian approach and produces unbiased results.
>> 
>> See papers by David Mackay on this topic and eg. page 6-8 of the
>> chapter on 'Hierarchical Models' in the SPM book (this is available
>> under publications/book chapters on my web page 
>> http://www.fil.ion.ucl.ac.uk/~wpenny/ - note gamma and (k-gamma)
>> terms in denominator of eqs 32 and 35 denoting the partitioning of
>> the degrees of freedom).
>> 
>> Nevertheless, I'd like to read page 1426 of your introductory
>> paper. Can you send it to me ?
>> 
>> Best wishes,
>> 
>> Will.
>> 
>> In particular, the outcome of the
>>>> iterations may become biased toward the original source
>>>> covariance used. In my test application of the described EM
>>>> algorithm I found that scaling the original source covariance
>>>> matrix changes the resulting sources estimate, which, in
>>>> principle, should not happen. For comparison, this problem does
>>>> not occur, when the source covariance parameters are learned
>>>> using ordinary or general cross-validation (OCV or GCV).
>>>> 
>>>> Best, Yury
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>> -- William D. Penny Wellcome Trust Centre for Neuroimaging 
>> University College London 12 Queen Square London WC1N 3BG
>> 
>> Tel: 020 7833 7475 FAX: 020 7813 1420 Email:
>> [log in to unmask] URL: http://www.fil.ion.ucl.ac.uk/~wpenny/
>> 
>> 
> 

-- 
William D. Penny
Wellcome Trust Centre for Neuroimaging
University College London
12 Queen Square
London WC1N 3BG

Tel: 020 7833 7475
FAX: 020 7813 1420
Email: [log in to unmask]
URL: http://www.fil.ion.ucl.ac.uk/~wpenny/

FW: Interesting Article

At the end of this article, the claim about "internet affects memory" -- Use myself as an example, I did feel the changes in my memory (loss) after I had computer and internet.

Read this article (interesting! share in my blog):

http://www.smartplanet.com/blog/science-scope/scientists-figured-out-why-we-cant-get-smarter/9631

 

Scientists figured out why we can’t get smarter

By Boonsri Dickinson | August 1, 2011, 12:12 PM PDT

 

Here’s an interesting fact: Smart people have faster impulses in the brain than less intelligent people. That’s all according to one Cambridge professor by the name of Ed Bullmore. But as far as getting any smarter, tough luck. British scientists made a convincing case for why our brains have reached full capacity: Human brains would consume too much energy.
Simon Laughlin, professor of neurobiology, at Cambridge University told The Sunday Times: ‘We have demonstrated that brains must consume energy to function and that these requirements are sufficiently demanding to limit our performance and determine design.”
There’s a chance the human brain could start to conserve energy and bring us back towards the size of the noggins of our Neanderthal ancestors. The researchers took into account the structure of the brain and figured out how much energy brain cells consume.
Mathematically speaking, the brain is an energy hog. It’s physically smaller than the rest of the human body, yet it consumes 20 percent of our energy. Energy is needed to fire electrical impulses so neurons can communicate with each other and also maintain the health of the cells to keep the tissues in the brain alive.
To get any smarter, the brain would need extra energy and oxygen, something all the coffee and Red Bull in the world probably can’t provide. Also, in the The Sunday Times story, the researchers say there’s a link between how connected different brain areas are and IQ. However, there isn’t enough energy to keep up with any increase in brain power.
Say it ain’t so that brain connections can’t get much better than this. Perhaps, this is as good as it gets.
With the way things are going with the Internet, maybe we can off-load some of the work onto computers and save some energy. If you recall, a recent study showed that the Internet affects your memory. On the upside, we’ve been able to overcome energy hurdles when building computers, so maybe there’s a chance we can do the same for human brains. If not, then scientists can always try to use machines to augment human intelligence or the other way around.
According to a recent Time magazine feature:

46 years later, Kurzweil believes that we’re approaching a moment when computers will become intelligent, and not just intelligent but more intelligent than humans. When that happens, humanity — our bodies, our minds, our civilization — will be completely and irreversibly transformed. He believes that this moment is not only inevitable but imminent. According to his calculations, the end of human civilization as we know it is about 35 years away.

via The DailyMail but the story originally appeared in The Sunday Times

Photo: NSF

http://www.creativitypost.com/psychology/the_brain_as_a_network_focusing_your_network

Brain scan study to understand workings of teenage mind: http://www.bbc.co.uk/news/health-22510866

http://www.creativitypost.com/psychology/the_brain_as_a_network_focusing_your_network

irrelevant notes:  http://www.jobs.cam.ac.uk/job/?category=2

August 26 - Fifth year of Ph.D. study

Autumn Semester of Fifth year Ph.D. study will start on August 26.  Time really flies.  Soon, I will graduate and own my Ph.D. in Biomedical Engineering and start another journey in academia.

Look back, I really appreciate all the great people I encountered and thanks God for his blessings to be surrounded with great leaders and wonderful opportunities.

Without my mentors (Drs. Scott Holland and Christy Holland, Jerzy Szaflarski, Jarek Meller, Weihong Yuan, Jennifer Vannest, Jing Xiang, Tzipi Horowitz-Kraus, etc), I would not be able to accomplish what I have accomplished.  The whole experience of get your Ph.D. is irreplaceable.

My take home message:  You need to be eager to learn, be a good team-player, and always have a positive attitude.  Remember, you might not be able to change or control your circumstance, but you can definitely control your response to the circumstance.  Good luck with everyone who is considering of pursing a Ph.D. or is currently pursing Ph.D.  Hang in there.  A big rainbow is waiting for you at the end of the P.H.D journey.

After Ph.D.

Good resource: http://studentaffairs.stanford.edu/cdc

Upgrade ASUS x202e

Recently, I got ASUS x202e with discover points award.  This cute pink touch-screen laptop only cost me 360 bucks from Amazon.com.  (http://www.amazon.com/gp/product/B00B7JQQQA/ref=oh_details_o04_s00_i00?ie=UTF8&psc=1)

Technical Details

Summary

Screen Size	11.6 inches
Max Screen Resolution	1366x768 pixels
Processor	1.8 GHz Core i3-3217U
RAM	4 GB DDR3
Hard Drive	500 GB
Graphics Coprocessor	Intel GMA HD
Wireless Type	802.11bgn
Number of USB 2.0 Ports	2
Number of USB 3.0 Ports	1

Other Technical Details

Brand Name	Asus
Series	ASUS VivoBook
Item model number	ASUS VivoBook X202E-DH31T-PK
Hardware Platform	PC
Operating System	Windows 8
Item Weight	2.9 pounds
Item Dimensions L x W x H	11.90 x 7.90 x 0.80 inches
Color	Pink
Processor Brand	Intel
Processor Count	2
Computer Memory Type	DDR3 SDRAM
Hard Drive Interface	Serial ATA-300
Hard Drive Rotational Speed	5400 RPM
Batteries	1 Lithium ion batteries required. (included)

I really like the color and solid build of this cute laptop.  The System is a little slow.  I decided to upgrade the hard disk to SSD and the wireless card to Intel 6235AN.

(1) Samsung MZ-7TD250BW 840 Series Solid State Drive (SSD) 250 GB Sata 2.5-Inch (169.99 bucks)
(2) Intel Network 6235AN.HMWWB Centrino WiFi Card Advanced-N 6235 Dual Band Bluetooth (22 bucks)
(3) Windows 8 Pro 64-bit student version from school (8 bucks)

~200 bucks in total for upgrading cost.

Steps for installation of SSD hard disk:
[1] unscrew the 9 screws in the bottom

[2] you need an old credit card to open the lid (see figure)

[3] unscrew the 3 screws which hold the hard disk.












[4] you can remove the black foam and put on the SSD hard disk, or you don't have to do that since SSD is not as fragile as the regular hard disk.









Steps for installation of new Intel 6235AN wireless card (will boost the surf speed):

[1] unplug the white and black wires and unscrew 1 screw, the card will pop up itself

[2] put the new one in the place, black-main, white-AUX

















Bingo!

Done!

Use external DVD player and install the clean windows 8 pro 64-bit.  Very fast installation.

This upgrade made this laptop boots much fast.  I love it and enjoy the great deal.

ASUS x202e -  A budget touch-screen (11.6 inch) laptop! - highly recommended to those who want to find a light-weighted and touch-screen laptop.

Much cheaper than Mac Air or Dell XPS 12.

If you don't have tight budget, I would recommend you to look into DELL XPS 12 - Cool design unit.

Hope the information is helpful to you!