Sampling from Gaussian mixture models, when are the sampled data independent?Central moments of a gaussian mixture density?Whitening a mixture of GaussiansGaussian mixture regression in higher dimensionsGMM weighting change during marginalizationdoubt regarding the gaussian mixture model definitionDecompose/split a single multivariate gauss into random gaussian mixtureConditional distribution for Gibbs sampling for Gaussian mixturePractical considerations on a mixture of Multivariate Normals, with many termsCan we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?Why use a Gaussian mixture model?
What is the purpose of a disclaimer like "this is not legal advice"?
Giving a career talk in my old university, how prominently should I tell students my salary?
Translation of 答えを知っている人はいませんでした
What is better: yes / no radio, or simple checkbox?
Finding the minimum value of a function without using Calculus
Was it really inappropriate to write a pull request for the company I interviewed with?
Graphic representation of a triangle using ArrayPlot
Would those living in a "perfect society" not understand satire
Are small insurances worth it?
Writing text next to a table
Is this Paypal Github SDK reference really a dangerous site?
Should we avoid writing fiction about historical events without extensive research?
Why aren't there more Gauls like Obelix?
How to install round brake pads
Create chunks from an array
Why is there an extra space when I type "ls" on the Desktop?
Converting from "matrix" data into "coordinate" data
The (Easy) Road to Code
ESPP--any reason not to go all in?
Did Amazon pay $0 in taxes last year?
How to educate team mate to take screenshots for bugs with out unwanted stuff
Too soon for a plot twist?
"If + would" conditional in present perfect tense
Why does Central Limit Theorem break down in my simulation?
Sampling from Gaussian mixture models, when are the sampled data independent?
Central moments of a gaussian mixture density?Whitening a mixture of GaussiansGaussian mixture regression in higher dimensionsGMM weighting change during marginalizationdoubt regarding the gaussian mixture model definitionDecompose/split a single multivariate gauss into random gaussian mixtureConditional distribution for Gibbs sampling for Gaussian mixturePractical considerations on a mixture of Multivariate Normals, with many termsCan we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?Why use a Gaussian mixture model?
$begingroup$
Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions
$p(x) = sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)$
where $w_i$ are the weights.
Now I sample some points $x_n$ from $p(x)$
What condition is needed to ensure that the points are independent from each other?
Is it sufficient to assume that each component $mathcalN(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?
normal-distribution mixed-model random-variable independence gaussian-mixture
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
$endgroup$
add a comment |
$begingroup$
Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions
$p(x) = sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)$
where $w_i$ are the weights.
Now I sample some points $x_n$ from $p(x)$
What condition is needed to ensure that the points are independent from each other?
Is it sufficient to assume that each component $mathcalN(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?
normal-distribution mixed-model random-variable independence gaussian-mixture
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
$endgroup$
add a comment |
$begingroup$
Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions
$p(x) = sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)$
where $w_i$ are the weights.
Now I sample some points $x_n$ from $p(x)$
What condition is needed to ensure that the points are independent from each other?
Is it sufficient to assume that each component $mathcalN(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?
normal-distribution mixed-model random-variable independence gaussian-mixture
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
$endgroup$
Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions
$p(x) = sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)$
where $w_i$ are the weights.
Now I sample some points $x_n$ from $p(x)$
What condition is needed to ensure that the points are independent from each other?
Is it sufficient to assume that each component $mathcalN(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?
normal-distribution mixed-model random-variable independence gaussian-mixture
normal-distribution mixed-model random-variable independence gaussian-mixture
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
asked 7 hours ago
Shamisen ExpertShamisen Expert
126112
126112
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)tag1$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_11$ is closer to $mu_1k$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
$$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcalN(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcalN(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.
answered 7 hours ago
Xi'anXi'an
57.9k897360
$endgroup$
$begingroup$
Thanks for your answer. What is the meaning of $mathcalM$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
$endgroup$
– Shamisen Expert
7 hours ago
$begingroup$
Xi'an is using $mathcalM$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
$endgroup$
– mb7744
43 mins ago
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396582%2fsampling-from-gaussian-mixture-models-when-are-the-sampled-data-independent%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)tag1$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_11$ is closer to $mu_1k$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
$$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcalN(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcalN(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.
answered 7 hours ago
Xi'anXi'an
57.9k897360
$endgroup$
$begingroup$
Thanks for your answer. What is the meaning of $mathcalM$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
$endgroup$
– Shamisen Expert
7 hours ago
$begingroup$
Xi'an is using $mathcalM$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
$endgroup$
– mb7744
43 mins ago
add a comment |
$begingroup$
You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)tag1$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_11$ is closer to $mu_1k$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
$$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcalN(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcalN(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.
answered 7 hours ago
Xi'anXi'an
57.9k897360
$endgroup$
$begingroup$
Thanks for your answer. What is the meaning of $mathcalM$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
$endgroup$
– Shamisen Expert
7 hours ago
$begingroup$
Xi'an is using $mathcalM$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
$endgroup$
– mb7744
43 mins ago
add a comment |
$begingroup$
You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)tag1$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_11$ is closer to $mu_1k$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
$$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcalN(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcalN(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.
answered 7 hours ago
Xi'anXi'an
57.9k897360
$endgroup$
You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_i = 1^N w_i mathcalN(x;mu_i, Sigma_i)tag1$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_11$ is closer to $mu_1k$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
$$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcalN(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcalN(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.
answered 7 hours ago
Xi'anXi'an
57.9k897360
edited 6 hours ago
answered 7 hours ago
Xi'anXi'an
57.9k897360
answered 7 hours ago
Xi'anXi'an
57.9k897360
answered 7 hours ago
Xi'anXi'an
57.9k897360
57.9k897360
$begingroup$
Thanks for your answer. What is the meaning of $mathcalM$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
$endgroup$
– Shamisen Expert
7 hours ago
$begingroup$
Xi'an is using $mathcalM$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
$endgroup$
– mb7744
43 mins ago
add a comment |
$begingroup$
Thanks for your answer. What is the meaning of $mathcalM$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
$endgroup$
– Shamisen Expert
7 hours ago
$begingroup$
Xi'an is using $mathcalM$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
$endgroup$
– mb7744
43 mins ago
$begingroup$
Thanks for your answer. What is the meaning of $mathcalM$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
$endgroup$
– Shamisen Expert
7 hours ago
$begingroup$
Thanks for your answer. What is the meaning of $mathcalM$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
$endgroup$
– Shamisen Expert
7 hours ago
$begingroup$
Xi'an is using $mathcalM$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
$endgroup$
– mb7744
43 mins ago
$begingroup$
Xi'an is using $mathcalM$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
$endgroup$
– mb7744
43 mins ago
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396582%2fsampling-from-gaussian-mixture-models-when-are-the-sampled-data-independent%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
